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ABSTRACT 


The  purpose  behind  this  thesis  was  two-fold.  First, 
the  effect  of  sound  delivery  on  a  user' s  sense  of  presence 
in  a  virtual  environment  was  investigated.  Second,  the 
physiological  responses  of  electrodermal  activity,  heart 
rate,  and  temperature  were  measured  and  correlated  to  the 
user's  subjective  sense  of  presence  in  an  attempt  to 
determine  if  physiological  measures  can  be  used  in  the 
future  as  an  objective  measure  of  presence. 

A  computer  based  first-person  shooter  game  (Medal  of 
Honor:  Allied  Assault™)  was  utilized  as  the  virtual 
environment.  The  independent  variable  was  sound  delivery 
method  (no  sound,  5.1  surround  sound,  headphones,  and 
headphones  with  subwoofer) .  The  dependent  variables  were 
physiological  response  and  questionnaire  results. 

Results  indicated  that  sound  contributed  to  the  user' s 
sense  of  presence  as  evidenced  by  electrodermal  activity 
and  temperature  and  questionnaire  scores.  Also, 
significant  changes  occurred  between  the  speaker  and 
headphone  sound  delivery  methods.  This  response  suggests 
that  speakers  created  a  higher  sense  of  emotion  and 
possibly  induced  a  higher  level  of  presence  in 
participants . 
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I. 


INTRODUCTION 


The  United  States  military  has  been  using  virtual 
environments  for  many  years.  Examples  include  flight 
simulators  for  pilot  training  and  conduct  of  fire  trainers 
for  tanks  and  armored  vehicles.  As  resources  dwindle,  both 
monetarily  and  environmentally,  the  military  has  continued 
to  look  to  technology  to  overcome  the  lack  of  resources  for 
training . 

The  first  virtual  environment  trainers  were  designed 
for  mounted  soldiers  and  airmen,  those  who  operate  vehicles 
such  as  tanks  and  aircraft.  But  with  dwindling  resources, 
the  emphasis  is  shifting  to  simulation  of  all  types  of 
military  operations  including  dismounted  operations.  These 
new  virtual  environments  have  the  potential  of  allowing 
safer  and  less  expensive  training  for  the  dismounted 
military  [LAMP  94]  .  While  technology  such  as  virtual 
environments  is  not  the  panacea  for  all  training,  it  does 
have  its  applications. 

A  motto  in  the  military  is  "train  like  we  fight,  fight 
like  we  train".  Using  real  world  exercises,  it  is  easy  to 
see  how  the  motto  is  supported.  However,  when  the  training 
is  done  in  a  virtual  environment,  it  may  be  difficult  to 
adhere  to  the  motto.  In  order  for  the  military  to  "train 
like  we  fight"  the  soldier  might  benefit  from  "feeling" 
like  he  is  in  the  environment.  This  is  commonly  referred 
to  as  presence  or  immersion.  But  until  an  objective 
measure  of  presence  is  discovered,  it  is  hard  to  determine 
what  effect  presence  has  on  training. 
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A.  PRESENCE  IN  VIRTUAL  ENVIRONMENTS 

One  aspect  of  the  subjective  experience  in  VE 
systems  that  has  received  considerable  attention 
is  the  extent  to  which  the  human  operator  loses 
his  awareness  of  being  present  at  the  site  of  the 
interface  and  instead  feels  present  in  the 
artificial  environment.  This  feature  is  often 
referred  to  under  the  headings  of  tele-presence, 
virtual  presence,  or  synthetic  presence  and  is 
dependant  on  many  factors,  including  (but  not 
limited  to)  the  extent  to  which  the  interface  is 
transparent  and  attenuates  stimulation  from  the 
immediate  environment,  as  well  as  the  amounts  and 
kinds  of  interaction  that  take  place  in  the 
artificial  environment  [DURL  95]. 

The  term  "virtual  environment"  or  "virtual  reality" 
implies  the  user  will  experience  an  alternate  reality  in 
place  of  true  reality.  Within  this  context,  the  term 
"presence"  has  been  used  to  describe  the  subjective  state 
of  "being  there"  in  a  mediated  environment.  The  reason  for 
the  perception  of  being  in  the  environment  is  both  a 
product  of  the  environment  and  the  interface  used  to 
interact  with  that  environment.  Witmer  and  Singer  consider 
presence  and  immersion  inseparable  entities  [WITH  98]. 
Slater  considers  immersion  a  product  of  the  hardware  used 
in  the  virtual  environment  [SLAT  99]  .  The  difficulty  lies 
in  measuring  "presence". 

A  better  understanding  of  what  presence  is,  what 
encourages  and  discourages  it  in  users,  and  its 
effects,  should  save  valuable  time  and  money  and 
improve  the  end  product  in  the  design  of  new  and 
redesign  of  current  media  technologies  [LOME  97] . 

Even  though  presence  has  not  been  adequately  defined,  it  is 
generally  believed  that  the  greater  number  of  human  senses 
involved  in  a  virtual  environment,  the  greater  the 
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possibility  the  VE  has  of  generating  presence  [LAWS  98]  . 
This  line  of  thinking  follows  the  real  world.  People  are 
present  in  the  real  world  and  are  inundated  with  stimuli 
that  affect  all  of  their  senses.  In  order  to  achieve  the 
same  sense  of  presence  in  the  real  world,  the  virtual  world 
should  incorporate  as  many  of  the  same  senses  as  possible. 

B .  QUANTIFYING  PRESENCE 

Research  in  quantifying  presence  is  still  in  its 
infancy.  Various  means  have  been  used  in  an  effort  to 
quantify  this  phenomenon.  Two  categories  of  measurement 
methods  (subjective  and  objective)  have  been  used  in  an 
attempt  to  quantify  presence.  Due  to  the  subjective  nature 
of  presence,  it  is  most  easily  measured  using  subjective 
questionnaires.  This  method  allows  the  user  to 
subjectively  rate  how  present  they  felt  in  the  environment 
by  answering  a  list  of  questions  concerning  their  sense  of 
presence  while  in  the  VE .  However,  there  are  inherent 
problems  with  subjective  questionnaires.  Some  drawbacks 
include  the  respondent's  understanding  of  presence,  relying 
on  memory  to  answer  the  questionnaire,  and  bias  due  to  the 
appeal  of  the  VE  content.  Other  subjective  means  of 
measuring  presence  are  continuous  presence  assessment  and 
psychophysical  methods.  Objective  corroborative 
measurement  methods  include  collecting  data  on  a  user' s 
postural  changes  during  VE  exposure,  measuring  the  user's 
physiological  responses  during  VE  exposure,  and 
administering  a  post -exposure  test  used  to  quantify  where 
attentional  resources  were  focused  during  exposure  to  a 
dual-task  environment.  All  of  these  methods  are  discussed 
in  greater  detail  in  the  background  section. 
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C.  RESEARCH  OBJECTIVES 

The  first  objective  of  this  research  is  to  determine 
how  the  method  of  sound  delivery  affects  the  user's  sense 
of  presence  in  a  virtual  environment.  The  motivation  for 
this  is  the  continuing  dependence  of  the  military  on 
computer-based  training  aids.  Headphone  systems  have  a 
much  smaller  footprint  than  speaker  systems,  making  them  a 
desirable  choice  for  deployable  simulators.  If  it  can  be 
determined  that  headphone  delivery  is  as  effective  as 
speakers  in  creating  a  high  sense  of  presence,  then 
military  organizations  will  be  able  to  minimize  cost  and 
space  requirements  for  computer  based  training  aids. 
Presence  will  be  measured  using  both  objective  and 

subjective  means. 

Another  goal  is  to  investigate  an  objective  method  for 
measuring  a  user' s  sense  of  presence  in  a  virtual 
environment.  Current  methods  rely  on  subjective  ratings 

that  may  vary  widely  between  individuals  as  well  as  the 

same  individual  between  rating  periods.  Objective  measures 
remove  some  of  this  variability  and  may  allow  for  a  more 
reliable  measurement  of  presence.  The  operational 

definition  of  presence  for  this  study  is  change  in 

physiological  responses.  The  participant's  physiological 
responses  will  be  recorded  while  they  are  exposed  to  and 
interact  with  the  virtual  environment.  These  responses 
will  give  insight  into  the  emotional  state  of  the 
participant  while  in  the  virtual  environment.  The  change 
in  emotional  state  should  also  result  in  a  change  in  the 
user's  sense  of  presence  (see  figure  1) .  Additionally,  the 
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responses  will  be  compared  to  the  participant's  subjective 
level  of  presence  as  measured  with  a  questionnaire  to 
determine  if  a  correlation  exists. 


Figure  1.  Theoretical  Approach  Linking  Emotional  State  to 

Presence . 


D.  THESIS  ORGANIZATION 

This  thesis  is  organized  into  the  following  chapters: 


•  Chapter  I:  Introduction.  This  chapter  provides 
an  overall  outline  of  the  thesis.  It  discusses 
the  importance  of  presence  in  virtual 
environments  and  current  measuring  techniques. 

It  also  covers  the  research  objectives  and 
motivation  behind  this  thesis. 

•  Chapter  II:  Background.  This  chapter  covers  past 
and  present  work  relevant  to  this  thesis.  It 
defines  presence  and  immersion,  differences 
between  the  two,  factors  that  influence  presence, 
methods  for  measuring  presence,  and  describes  how 
physiological  monitoring  can  be  used  to  link 
subjective  measures  of  presence  (questionnaires) 
to  objective  measures  of  presence  (physiological 
responses) .  It  also  discusses  the  background  on 
emotions  and  the  physiological  correlates  of 
emotion . 
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•  Chapter  III:  Method:  This  chapter  describes  the 
experiment  and  the  four  treatments  administered 
to  the  participants. 

•  Chapter  IV:  Analysis.  This  chapter  contains  the 
results  of  the  experiment  in  terms  of  the  stated 
hypotheses . 

•  Chapter  V:  Conclusions  and  Recommendations.  This 
chapter  provides  an  overview  of  the  experiment, 
the  conclusions,  and  recommends  future  work  in 
this  area  of  research. 

•  Appendices: 

A.  Raw  Data 

B.  Experiment  Protocol 

C.  In-brief  Script 

D.  Consent  Forms 

E.  Questionnaires 

F.  Equipment  Specifications 
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II .  BACKGROUND 


A .  VIRTUAL  ENVIRONMENTS 

Over  the  past  couple  of  decades,  virtual  reality  (VR) 
and  virtual  environments  (VE)  have  become  hot  topics  of 
discussion  and  experimentation.  Virtual  reality  has 
enabled  us  to  create  many  different  virtual  objects  and 
environments.  Virtual  universities,  offices,  pets,  actors, 
studios,  museums,  doctors,  and  wind  tunnels  are  some  of  the 
many  virtual  creations,  just  to  name  a  few.  Virtual 
environments  or  synthetic  environments  create  a  virtual 
reality  using  iunmersive  hardware  devices  designed  to  allow 
the  user  to  navigate  and  interact  with  the  environment  in 
such  a  way  that  he  actually  feels  present  within  the 
environment.  In  order  to  fully  understand  this  definition, 
it  is  imperative  that  we  understand  how  virtual  reality, 
immersion,  and  presence  relate  to  virtual  environments. 

To  adequately  define  virtual  reality,  we  must  first 
understand  the  meanings  of  the  words  "virtual"  and 
"reality".  Virtual  is  defined  as  existing  or  resulting  in 
effect  or  essence  though  not  in  actual  fact,  form,  or  name. 
Reality  is  defined  as  the  quality  or  state  of  actual  or 
true  [WEBS  88]  .  Combining  these  two  definitions,  virtual 
reality  seems  to  suggest  a  reality  that  is  believable 
(accepted  as  real  or  true)  ,  and  yet  does  not  physically 
exist . 

John  Vince  describes  two  types  of  VRs,  immersive  and 
non-immersive .  Immersive  VRs  were  originally  described  as 
environments  that  provided  the  user  with  a  "first  person" 
view  of  the  virtual  world. 
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In  the  1980s  real-time  computer  graphics  became  a 
reality,  and  VR  became  a  commercial  reality. 
Initially,  early  VR  systems  comprised  a  real-time 
computer  system,  a  head-mounted  display  (HMD) , 
and  an  interactive  glove.  Apart  from  supplying 
the  user  with  stereoscopic  images,  the  HMD 
immersed  the  user  in  the  virtual  world  by 
preventing  them  from  seeing  the  real  world. 
Immersion  increased  the  sensation  of  presence 
within  the  virtual  world,  and  for  some  people, 
immersion  distinguished  VR  systems  from  other 
types  of  real-time  computer  graphics  systems 
[VINC  01] . 

Early  computer  games  were  considered  non-immersive 

VRs .  They  did  not  provide  the  user  with  a  real-time  3D 

environment  in  which  to  navigate  or  interact,  and  they  did 

not  provide  a  "first  person"  view  of  the  virtual  world. 
Today,  an  immersive  VE  can  be  achieved  using  a  PC  capable 
of  displaying  real-time  images  of  3D  environments  that 
provide  a  "first  person"  view  and  allow  the  user  to 

navigate  and  interact  with  the  mediated  environment. 
Although  computer  simulations  and  "first  person"  type  games 
can  induce  a  sense  of  presence  in  their  users,  their 
immersive  capabilities  still  fall  well  short  of  their  more 
expensive  partners  (CAVES  and  fully  interactive  HMD 
environments) .  In  order  to  fully  understand  what  virtual 
reality  is,  discussion  of  its  defining  characteristics, 
immersion  and  presence,  follows. 

B .  IMMERSION 

Immersion,  as  defined  by  Vince,  is  the  sensation  of 
being  part  of  a  VE  [VINC  01]  .  Slater  defines  immersion 
more  specifically  as  a  product  of  the  VE  hardware  [SLAT 
99]  .  There  are  many  characteristics  of  VE  hardware  that 
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encourage  this  sensation  and  a  non-exhaust ive  list  of  some 
of  these  characteristics  and  how  they  influence  immersion 
is  shown  in  table  1 . 


Table  1.  Hardware  Characteristics  that  Influence 

Immersion . 


Immersive 

Characteristics 

Affected 

Sense 

Influence  on  Immersion 

Field  of  View 

vision 

Limits  Real-World  Distractions 

Stereoscopy 

Increases  Depth  Perception 

Display  Screen  Resolution 

Improves  Image  Quality 

Color  Display 

Increases  Scene  Realism 

Refresh  Rate 

Improves  Display  Quality 

Update  Rate 

Reduces  Flicker  Distractions/Eye 
Strain 

Sound 

Hearing 

Introduces  Sense  of  Hearing 

Surround  Sound 

Improves  Sense  of  Location  and 
Orientation  (Spatialization) 

Haptic/Tactile  Feedback 

Touch 

Introduces  the  Sense  of  Touch 

Provides  Force  Feedback 

Olfactory  Feedback 

Smell 

Introduces  the  Sense  of  Smell 

Kinesthetic  Feedback 

Vestibular 

Provides  a  Natural  Means  for 
Self-Motion/Navigation  in  the  VE 

Lessiter  claims  "...the  greater  the  number  of  sensory 
inputs  provided  to  different  modalities,  the  greater  the 
sense  of  presence"  [LESS  01] .  This  can  be  restated  as  the 
greater  immersive  characteristics  of  the  environment  and 
the  more  modalities  presented  within  the  VE,  the  greater 
the  resulting  sense  of  presence  for  the  VE  participant. 
The  two  senses  that  receive  the  most  attention  in  VEs  are 
vision  and  hearing.  The  sense  of  vision  is  relied  upon 
about  70  percent  of  the  time  for  perception  in  the  real 
world,  while  hearing  (sound)  accounts  for  about  20  percent. 
The  remaining  ten  percent  is  distributed  among  the  other 
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sensing  modalities  [HEIL  92]  .  This  being  said,  Tom  Holman 
and  the  artists  at  Lucasfilm™  consider  audio  a  primary 
emotion-inducing  medium  in  movie  presentation  comprising  at 
least  50%  of  the  motion  picture  experience  [HOLM  00]  .  The 
above  table  shows  various  immersive  characteristics  that 
can  be  adjusted  to  influence  the  immersive  qualities  of  a 
VE .  As  the  immersive  qualities  of  a  VE  increase,  the 
ability  of  the  VE  participant  to  feel  like  they  are 

actually  present  in  the  environment  also  increases.  This 
feeling  of  actually  being  present  in  a  VE,  known  as 

"presence",  is  the  next  subject  of  discussion. 

C .  PRESENCE 

When  the  curtain  swept  up  to  reveal  the  now¬ 
legendary  wide-screen  roller  coaster  ride,  I 
realized  the  film's  creators  were  no  longer 
content  to  have  me  look  at  the  roller  coaster  but 
were  trying  to  put  me  physically  on  the  ride. 

The  audience  no  longer  surrounded  the  work  of 
art;  the  work  of  art  surrounded  the  audience 
just  as  reality  surrounds  us.  The  spectator  was 
invited  to  plunge  into  another  world.  We  no 
longer  needed  the  device  of  identifying  with  a 
character  on  the  other  side  of  the  'window.'  We 
could  step  through  it  and  be  a  part  of  the 
action!  -  Morton  Heilig  commenting  on  his 

experience  with  Cinerama  in  New  York,  1952. 

1 .  Definition  of  Presence 

The  most  commonly  stated  definition  of  presence  is  a 

sense  of  "being  there"  in  a  mediated  environment.  The  term 

implies  that  those  who  are  using  a  virtual  environment 

actually  feel  as  if  they  are  part  of  the  virtual 

environment  [KALA  00]  .  This  in  turn  implies  that  the  user 

will  get  as  much  sensory  input  from  the  virtual  environment 

as  he  would  from  the  real  world.  While  current  technology 
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cannot  support  the  last  statement  fully,  it  is  the  ideal 
virtual  environment. 

Presence  can  be  illustrated  as  a  "continuum  of 
reality".  On  this  continuum,  the  left  endpoint  is  zero  or 
low  presence  (less  realistic)  while  the  right  endpoint  is 
high  presence  (more  realistic,  see  figure  2) .  Consider  the 
following  environments  and  where  they  might  go  on  the 
continuum:  TV  show,  movie  (in  a  theater) ,  desktop  VR,  IMAX 
movie,  CAVE  application,  and  a  Head  Mounted  Display  (HMD) 
application.  The  TV  show  will  go  toward  the  low  end  of  the 
continuum.  While  placement  is  arbitrary,  TV  shows  do  not 
offer  much  in  the  way  of  viewer  involvement.  Next, 

consider  a  movie  shown  in  a  movie  theater.  It  offers  a 
more  compelling  environment  due  to  soundtrack  delivery  over 
a  high  quality  speaker  system  and  will  most  likely  induce  a 
higher  level  of  presence  than  the  TV  show.  Therefore,  it 

will  appear  to  the  right  of  the  TV  show  on  the  reality 

continuum.  Thirdly,  a  desktop  VR  application  allows  for 
user  interaction,  thereby  possibly  increasing  presence.  It 
will  go  to  the  right  of  the  movie.  An  IMAX™  movie,  on  the 
other  hand,  will  go  to  the  right  of  the  desktop  VR  due  to 
its  ability  to  involve  the  viewer  with  a  wider  field  of 
view.  Virtual  environments  that  involve  a  CAVE  or  a  HMD  go 
even  further  to  the  right.  The  CAVE  is  to  the  left  of  the 
HMD  due  to  the  HMDs  ability  to  completely  block  any 
external  visual  stimuli  from  the  user.  The  CAVE  still 
allows  the  user  to  see  the  floor,  back  wall,  or  any  other 
part  of  the  apparatus  that  is  not  part  of  the  display. 

Another  aspect  of  these  "environments"  that  needs  to 

be  considered  is  the  content  of  the  environment.  A  black 
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and  white  or  silent  movie  will  most  likely  rank  lower  in 
presence  than  the  new  "high-tech"  movies  of  today  (e.g. 
Star  Wars,  Terminator,  etc.) .  While  there  is  some  room  for 
argument  on  what  goes  where  on  the  continuum,  these  are 
just  examples  to  illustrate  the  concept.  There  are  no  hard 
and  fast  rules  on  placement  of  environments  on  the 
continuum  primarily  due  to  the  lack  of  a  solid  definition 
of  presence. 


Low 

Presence 

TV  Movie  Desktop  VR  IMAX  CAVE  HMD 

◄ - - - - - - 


High 

Presence 


► 


Figure  2.  Continuum  of  Reality. 
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A  number  of  variables  appear  as  possible  causes  of 
presence,  either  positively  or  negatively,  and  is  shown  in 
table  2 . 


Table  2.  Possible  Causes  of  Presence  (From:  [KALA  00]). 


Variable 

Contribution 

Form  Variables  -  This  group  includes 

the 

more  objective  parameters 

Sensory  outputs 

Number  of  sensory  outputs 

Positive 

(for  higher  numbers) 

Consistency  of  sensory  outputs 

Positive 

(when  consistent) 

Visual  outputs  —  have  various  dimensions 

Strong  - 

see  dimensions  below 

Display  size 

Positive 

(for  larger  proportion) 

Viewing  distance 

Positive 

(for  larger  proportion) 

Quality  of  image 

Positive 

(for  high  quality) 

Depth  cues 

Positive 

Camera  techniques 

Positive 

Audible  outputs  -  has  different  dimensions  Strong 

Other  sensory  outputs  (smells,  touch  etc 

.)  Can  be  influential,  but 

usually  less  strong  than 
audio  or  visual 

Body  movement  and  force  feedback 

Positive 

when  done  well 

Inactivity  of  medium 

Positive 

Visibility/obtrusiveness  of  medium 

Negative 

Interference  from  real  world 

Negative 

Human  contact 

Positive 

Content  variables  -  can  be  objective 

and 

subjective 

Characters  and  storylines 

Positive 

and  negative 

Media  conventions 

Usually 

negative 

Nature  of  representation 

Positive 

and  negative 

Media  user  variables  -  These  are  highly  subjective  and  depend  directly  on 
the  individual 

Willingness  to  suspend  disbelief 

Positive 

Previous  experience 

Positive 

or  negative 

13 


Although  there  exist  many  definitions  of  presence, 
Heeter  has  defined  presence  as  having  three  dimensions: 
personal,  social,  and  environmental  [HEET  92]. 

a .  Personal  Presence 

Personal  presence  refers  to  the  extent 
individuals  feel  they  are  present  when  operating  in  a 
virtual  environment.  Research  in  this  area  attempts  to 
identify  what  features  are  needed  in  a  virtual  environment 
to  convince  users  they  are  in  another  world  [BERN  99] .  The 
reasons  that  users  may  feel  like  they  are  in  the  virtual 
world  are  many  but  could  be  as  simple  as  seeing  their  own 
hand  in  the  virtual  world. 

ib .  Social  Presence 

Social  presence  is  the  extent  to  which  other 
beings  also  exist  in  a  virtual  environment  and  react  to  the 
primary  user.  The  premise  is  that  if  other  people  exist  in 
the  virtual  world,  it  lends  more  credence  that  the  world 
exists.  The  people  can  either  be  other  humans  interacting 
with  the  virtual  environment  or  computer  generated  beings 
that  are  a  part  of  the  environment.  In  either  case,  the 
perception  that  someone  recognizes  the  user  in  the  VE 
creates  a  more  believable  environment  for  the  user. 

c.  Environmental  Presence 

Environmental  presence  is  the  extent  to  which  the 
environment  reacts  to  the  user.  The  ability  to  modify  an 
environment  has  been  proposed  as  a  key  component  of 
presence  [SHER  92]  .  Modification  of  an  environment  can  be 
as  simple  as  moving  a  chair  or  turning  on  a  light  with  a 
light  switch.  Environmental  presence  is  also  enhanced  when 
the  objects  in  the  VE  behave  similar  to  their  real  world 
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counterparts.  If  the  user  has  the  ability  to  do  something 
in  the  VE  that  cannot  be  done  in  the  real  world  (i.e.,  walk 
through  objects  like  walls  or  furniture) ,  their  sense  of 
presence  may  be  adversely  affected. 

2 .  Measuring  Presence 

Research  in  presence  measurement  is  still  relatively 
young.  Currently,  there  is  no  generally  accepted  theory  on 
presence  [IJSS  00]  .  In  an  effort  to  quantify  the  user's 
sense  of  presence,  most  measuring  techniques  contain  a 
means  for  collecting  data  on  a  combination  of  all  three 
dimensions  explained  above.  Because  of  this,  many  methods 
have  been  utilized.  The  different  methods  can  be  divided 
into  two  categories:  subjective  measures  and  objective 
corroborative  measures.  The  following  sections  discuss  the 
more  common  methods  used  in  each  category, 
a.  Subjective  Measures 

The  subjective  nature  of  presence  has  led  to  the 
argument  that  the  "subjective  report  is  the  essential  basic 
measurement"  for  obtaining  a  user's  level  of  presence  [SHER 
92] .  The  most  common  subjective  measure  is  through  post¬ 
test  rating  scales.  Other  methods  include  continuous 
presence  assessment  and  psychophysical  methods.  The  lack  of 
an  exact  definition  of  presence  makes  creating  one  standard 
subjective  means  for  gathering  data  nearly  impossible. 

(1)  Questionnaires:  Questionnaires  are  by 

far  the  easiest  method  available  for  measuring  presence. 

An  advantage  of  questionnaires  is  the  unobtrusive  way  in 

which  they  can  be  administered.  There  are  several 

subjective  questionnaires  designed  to  obtain  an 
individual's  sense  of  presence  [SLAT  98,  SING  96,  LESS  00] . 
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Although  most  questionnaire  authors  possess  similar  views 
on  presence,  the  lack  of  a  standard  operational  definition 
for  presence  allows  different  interpretations  on  how  to 
quantify  this  phenomenon. 

Slater's  definition  of  presence  includes  three 
aspects:  (i)  the  sense  of  being  in  the  mediated 
environment,  (ii)  the  extent  users  respond  to  events  in  the 
VE  vice  the  real  world,  and  (iii)  the  extent  that  user's 
remember  the  VE  as  a  place  they  visited  rather  than 
something  that  was  seen  on  the  computer  [SLAT  99]  .  Slater 
further  defines  immersion  as  the  extent  the  VE  hardware 
delivers  a  surrounding  environment,  shutting  out  real  world 
sensations  and  replacing  them  with  virtual  world 
sensations . 

Witmer  and  Singer,  on  the  other  hand,  consider 
presence  a  product  of  involvement  and  immersion. 
Immersion,  according  to  Witmer  and  Singer,  is  a 
psychological  state  characterized  by  the  perception  of 
being  enveloped  by,  included  in,  and  interacting  with  an 
environment  that  feeds  the  user  a  continuous  stream  of 
stimuli  [WITH  98]  .  It  is  this  difference  of  opinion  that 
separates  Slater's  questionnaire  from  Witmer  and  Singer's. 

Slater  takes  issue  with  Witmer  and  Singer' s 
questionnaire  because  some  of  their  questions  address 
hardware  issues  that  he  believes  is  a  factor  of  immersion 
not  presence.  I jsslestei jn,  de  Ridder,  Ereeman,  &  Avons 
[IJSS  00]  also  note  a  problem  in  Witmer  and  Singer's 
questionnaire  is  the  inseparability  of  individual 
characteristics  and  properties  of  the  virtual  environment 
caused  by  their  definition  of  immersion.  Various  other 
questionnaires  have  been  developed  and  are  being  used  in  an 
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effort  to  gauge  an  individual's  level  of  presence  when 
placed  in  a  mediated  environment,  but  limited  consolidated 
data  is  available  on  these  questionnaires. 

(2)  Continuous  assessment:  An  obvious 

criticism  of  post-test  ratings  is  the  lack  of  rating 
presence  as  a  variation  in  time.  The  result  of 

questionnaires  is  the  overall  sense  of  presence  experienced 
by  the  user  throughout  the  time  spent  in  the  virtual 
environment.  In  order  to  overcome  this  shortfall,  attempts 
have  been  made  to  measure  presence  on  a  more  continuous 
scale.  This  method  requires  the  user  to  make  a  presence 
judgment  during  use  of  the  environment  by  adjusting  a 
slider  that  is  continuously  sampled  by  the  computer  (see 
figure  3)  [IJSS  00] . 


Score 


Tlmt 


Figure  3.  Continuous  Assessment  Schematic  (From:  [IJSS  00]) 

Continuous  assessment  has  one  glaring  potential 
criticism:  how  does  adjusting  the  slider  affect  a  user's 
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sense  of  presence?  The  user  has  to  continuously  make 
"presence"  judgments  during  use  of  the  virtual  environment 
to  adjust  the  slider  thereby  potentially  disrupting  the 
whole  experience.  However,  this  may  not  be  the  case  since 
most  users  of  a  virtual  environment  know  that  they  are 
actually  in  a  lab  using  a  virtual  environment  and  reporting 
on  a  sensation  of  being  in  the  virtual  environment.  This 
is  common  to  both  continuous  assessment  and  questionnaires 
[IJSS  00] . 

(3)  Psychophysical  Methods:  Some 
psychophysical  methods  have  been  proposed,  but  not  much 
research  has  been  done.  Psychophysical  methods  include 
free-modulus  magnitude  estimation,  cross -modality  matching, 
and  paired  comparisons. 

Snow  and  Williges  used  free-modulus  magnitude 
estimation  to  investigate  the  effects  of  a  wide  variety  of 
parameters  on  presence  [SNOW  98].  Free-modulus  magnitude 
estimation  involves  the  presentation  of  several  stimuli  to 
a  participant.  The  participant  is  required  to  assign  a 
number  for  each  stimulus  based  on  the  strength  of  their 
subjective  sensation.  The  first  stimulus  presented  is 
almost  arbitrary,  but  the  following  stimuli  are  based  off 
the  first. 

The  second  psychophysical  method  is  cross-modality 
matching.  It  is  a  variation  of  magnitude  estimation  but  is 
usually  reserved  for  those  situations  where  verbal  scaling 
is  difficult.  Cross -modality  matching  requires  the 
participant  to  indicate  his  level  of  presence  in  one 
modality  by  expressing  adjustment  of  a  parameter  in  another 
modality  (e.g.  "Make  this  sound  as  intense  as  the  strength 
of  presence  you  experienced  in  this  VE")  [IJSS  00] . 
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The  final  psychophysical  method  to  discuss  is  paired 
comparisons.  Within  the  context  of  virtual  environments, 
this  has  sometimes  been  referred  to  as  the  "virtual  reality 
Turing  test"  [IJSS  00]  .  In  this  method,  a  participant  is 
asked  to  distinguish  a  real  scene  from  a  virtual  one. 
Obviously,  it  will  be  unlikely  that  the  participant  will 
not  be  able  to  determine  which  the  real  scene  is.  In  order 
to  give  each  scene  an  "equal  footing",  both  scenes  have  to 
be  viewed  under  the  same  constraints,  i.e.  limited  field  of 
view,  no  sound,  reduced  color/cont rast ,  etc.  The  amount  of 
degradation  of  the  real  scene  required  to  make  it 
indistinguishable  from  the  virtual  scene  is  the  measure  of 
presence.  A  criticism  of  this  method  is  that  it  is  not  a 
measure  of  presence  but  rather  a  measure  of  the 
participant's  ability  to  discriminate  between  two  bad 
images  [IJSS  00]. 

ib.  Objective  Corroborative  Measures 

One  of  the  downsides  of  using  subjective  measures 
of  presence  is  that  they  require  the  respondent  to  have  an 
idea  of  what  is  meant  by  presence.  Most  users  are 
unfamiliar  with  the  term  presence  as  it  applies  to  virtual 
environments.  Also,  if  a  participant  has  an  interest  in 
the  subject  being  displayed  in  the  environment,  his 
presence  score  may  be  biased  (see  [FREE  99] ) .  It  is  these 
reasons  that  an  objective  measure  of  presence  is  sought. 
There  are  three  main  objective  corroborative  measures  in 
use:  postural  responses,  dual  task  measures,  and 
physiological  responses. 

(1)  Postural  Responses:  Postural  responses 
are  a  measure  of  the  participant's  observed  body 
adjustments  during  virtual  environment  use.  The  idea  is 
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simple:  if  the  display  better  approximates  the  real  world, 
the  participant  will  react  as  if  he  is  in  the  real  world. 
This  method  bypasses  the  subjective  measures  by  measuring 
responses  that  are  automatic  and  not  deliberately 

controlled  [IJSS  00a]  .  In  a  study  where  participants  were 

asked  to  stand  as  still  as  possible  while  watching  a  video 
of  a  rally  car  circling  a  track,  it  was  found  that  postural 
responses  cannot  be  substituted  for  subjective  measures  of 
presence  since  the  two  measures  did  not  correlate  across 
participants.  However,  postural  responses  can  be  used  for 
corroboration  of  group  subjective  presence  ratings  [IJSS 
00a] .  Something  similar  to  postural  responses  is 

behavioural  response.  These  include  such  automatic  actions 
as  ducking  or  flinching.  The  premise  behind  behavioural 
responses  is  similar  to  that  of  postural  responses.  That 
is,  if  the  user  responds  to  the  virtual  environment  as  he 
would  in  the  real  world,  then  a  higher  sense  of  presence  is 
experienced . 

(2)  Dual  Task  Measures:  Dual  task  measures 
are  consistent  with  the  view  that  attention  allocation  is 
an  important  part  of  presence.  Because  of  this,  another 
objective  method  is  that  of  secondary  reaction  time 
measurements.  As  more  attention  is  focused  on  the  primary 
task,  less  attention  remains  for  the  secondary  task.  This 
typically  results  in  participants  making  more  errors  or 
taking  longer  to  respond  to  the  secondary  task  [IJSS  00]  . 
Darken,  Bernatovich,  Lawson,  and  Peterson  conducted  a  study 
comparing  the  roles  of  attention  and  spatial  comprehension 
on  presence.  They  concluded  that  attention  might  be  a 
reasonable  objective  measure  of  presence  but  found  no 
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correlation  between  spatial  comprehension  and  presence 
[DARK  99] . 

(3)  Physiological  Measures:  Physiological 
measures,  such  as  heart  rate  and  skin  conductance,  have 
been  suggested  as  objective  corroborative  measures  of 
presence.  However,  little  research  has  been  done  to  relate 
physiological  measures  to  presence  [IJSS  00]  .  In  order  to 
discuss  how  physiological  responses  are  related  to 
presence,  it  is  necessary  to  identify  the  relationship 
between  the  body's  physiological  responses,  emotions,  and 
presence.  The  next  section  discusses  previous  research  in 
determining  this  relationship  and  the  following  section 
discusses  the  theory  behind  emotions  and  how  they  relate  to 
physiological  responses. 

c.  Previous  Research  in  Measuring  Presence 

Although  research  in  the  measurement  of  presence 
is  in  its  infancy,  several  attempts  have  been  made  to 
establish  a  reliable  repeatable  measure  for  this 
phenomenon.  The  following  table  lists  several  research 
attempts,  identifies  the  independent  and  dependent 
variables,  lists  methods  of  measurement  used  during  these 
attempts,  and  the  results  from  these  efforts. 
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Table  3.  Presence  Related  Research. 


Researchers 

Independent  Variable 

Dependent 

Variable 

Method  of 

Measurement 

Results 

[DETE  98] 

Motion  -  Video  vs 

Still  Pictures 

Emotional 

Reactions 

Self-Reports 
(SR)  Phys  Data 
(EDA  &  HR) 

Motion  increased 
arousal  (EDA  &  SR) 
and  caused  HR 

deceleration 

[IJSS  00] 

Motion  -  Video  vs 
still  pictures 

Display  -  Mono  vs 

Stereo 

Postural 

Responses  (PR) , 
Presence  (P)  , 
Vection  (V) , 
Involvement  (I), 
and  Motion 

Sickness  (MS) 

Magnetic  Tracker 
Subjective 

Ratings 

PR  -  No  significant 
Response  (NSR) 

Motion  Increased  P, 

V,  &  I;  NSR  for  MS 

Display  Increased  P 

[DINH  98] 

Display  -  Low  vs  High 
Fidelity 

Audio  -  On  vs  Off 

Olfactory  -  On  vs  Off 

Tactile  -  On  vs  Off 

Presence  (P) 

Memory  (M) 

Qverall  P  rating 

Subjective  P 
questionnaire 

Quiz  on  Spatial 
Layout  (SL)  and 
Object  Location 
(OL) 

Display  had  no 
effect  on  P  or  M 

Audio  increased  P 
and  SL  M  but  had  no 
effect  on  OL  M 

Olfactory  slight 
increase  in  P  (Not 
Significant) , 
increased  OL  M 

Tactile  increase  in 

P  and  OL  M  but  no 
effect  on  SL  M 

[SLAT  95] 

Locomotion  -  walk  in 
place  or  pointed  via 
mouse 

Presence  (P) 

Nausea  (N) 

Association  with 
the  Virtual  Body 
(VB) 

Self-  Assessment 
Questionnaires 

Walking  - 

correlation  between 
higher  association 
with  the  VB  and 
higher  sense  of  P 

Pointing  -  resulted 
in  no  correlation 

Higher  degree  of  N 
associated  w/  higher 
sense  of  P 

[LESS  01] 

Speaker  Configuration 

EXP  1  -  1.0,  2.0,  5.1 

EXP  2  -  2.0,  2.1,  5.0, 
5.1 

Presence  (P) 

Audio-Visual  (AV) 
Quality 

Questionnaires 

EXP  1-5.1 
increased  P  &  AV 

EXP  2  -  Inclusion  of 
Bass  increased  P  & 

AV;  number  of 
channels  had  no 
effect 

[FREE  01] 

Speaker  Configuration 

2.0,  2.1, 

2 . 0 (Control) ,  5.0,  5.1 

Presence  (P) 

Audio-Visual  (AV) 
Quality 

Questionnaires 

Inclusion  of  Bass 
increased  P  &  AV; 
number  of  channels 
had  no  effect 

[SLAT  98] 

Body  Movement 

Task  Complexity 

Presence  (P) 

Questionnaire 

Increased  movement 
increased  sense  of  P 

Task  complexity  -  No 
Siqnif,  Effect 
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Table  3.  Presence  Related  Research (cent ) . 


Researchers 

Independent  Variable 

Dependent 

Variable 

Method  of 

Measurement 

Results 

[FREE  99] 

Motion  -  Observer 

Motion,  Scene  Motion, 
Minimal  Motion 

Display  -  Mono  vs 

Stereo 

Presence  (P) 

Interest  (I) 

Continuous 

Assessment 

EXP  1  -  Motion  and 
Display  increased  P 

EXP  2  -  No 

correlation  between 

I  and  P;  prior 
experience  in  the  VE 
affected  P 

EXP  3  -  Prior 
experience  affects 

P;  Stereo  Display 
increases  P 

[MEEH  00] 

#  of  VE  Exposures 

Presence  (P) 

Questionnaire 

Behavioral 
Responses 
(questionnaires 
and  video 
monitoring) 

Physiological 
Monitoring  (EDA, 
HR,  &  Temp) 

Correlation  between 
EDA  &  P 

Repeated  exposure 
decreases  P 

[DARK  99] 

EXP  1  -  Audio  & 

Display 

EXP  2  -  Audio 

Presence  (P) 

EXP  1  -  Attention 
(A) 

EXP  2  -  Spatial 

Comprehension 

(SC) 

Questionnaire 

Attention  quiz 

Spatial 

Comprehension 

quiz 

EXP  1  -  Audio 
increases 

engagement;  A  can  be 
used  as  a  partial 
measure  of  P 

EXP  2  -  Audio 
increases  P;  no 
correlation  between 

SC  &  P 

The  research  efforts  depicted  in  the  table  above  show 
that,  although  much  effort  in  establishing  a  clear 
measurement  criterion  for  presence  has  been  made,  few 
concrete  links  have  been  established  between  dependent 
variables  and  presence.  Additional  research  must  be 
accomplished  to  identify  the  variables  that  most  greatly 
affect  an  individual's  sense  of  presence. 

3 .  Emotions 

...when  the  mind  is  strongly  excited,  we  might 
expect  that  it  would  instantly  affect  in  a  direct 
manner  the  heart... when  the  heart  is  affected  it 
reacts  on  the  brain;  and  the  state  of  brain  again 
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reacts  through  the  pneumo -gastric  nerve  on  the 
heart;  so  that  under  any  excitement  there  will  be 
much  mutual  action  and  reaction  between  these, 
the  two  most  important  organs  of  the  body  [DARW 
72]  . 

Merriam-Webster  defines  emotion  as  (1)  the  affective 
aspect  of  consciousness;  (2)  a  state  of  feeling;  (3)  a 
psychic  and  physical  reaction  (as  anger  or  fear) 
subjectively  experienced  as  strong  feeling  and 
physiologically  involving  changes  that  prepare  the  body  for 
immediate  vigorous  action  [WEBS  88a]  .  The  earliest  and 
most  influential  theory  of  emotion  is  the  James-Lange 
theory.  This  classic  theory  in  psychology  originated  in 
the  1880s  independently  by  William  James,  an  American 
psychologist,  and  C.  G.  Lange,  a  Danish  physiologist. 
James  expressed  his  theory  as  follows: 

Common  sense  says,  we  lose  our  fortune,  are  sorry 
and  weep;  we  meet  a  bear,  are  frightened  and  run; 
we  are  insulted  by  a  rival,  are  angry  and  strike. 

The  hypothesis  here  to  be  defended  says  that  this 
order  of  sequence  is  incorrect, . . .  that  the  more 
rational  statement  is  that  we  feel  sorry  because 
we  cry,  angry  because  we  strike,  afraid  because 
we  tremble  [JAME  90] . 

The  shortened  version  of  his  statement  is  that  humans  feel 
emotional  because  we  sense  our  body  reacting.  Bodily 
reactions  and  the  perception  of  those  reactions  are  factors 
in  the  experience  of  the  emotion  [GRIN  78] .  The  flip  side 
is  that  without  bodily  reactions  there  would  be  no 
emotions . 

A  critic  of  the  James-Lange  theory  was  Walter  Cannon. 
In  a  critical  analysis  of  the  relevant  research  in  the 
early  1900s,  he  suggested  five  major  areas  of  objection  to 


the  James-Lange  theory  [GROS  67] .  Central  to  his  criticism 
of  the  theory  were  the  following  points  [GRIN  78] : 

1.  When  the  bodily  reactions  that  typically  occur  in 
emotion  are  prevented  from  occurring  (as  with 
transection  of  the  spinal  cord  and  vagus  nerve, 
and  removal  of  the  sympathetic  nervous  system) 
emotional  behavior  in  animals  is  not  visibly 
altered . 

2.  The  same  bodily  reactions  occur  in  all  emotions; 
hence  these  changes  cannot  produce  qualitatively 
different  emotions. 

3.  The  viscera  have  fewer  sensory  nerves  than  other 
structures;  hence  people  are  typically  unaware  of 
the  physiological  processes  occurring  there. 

4.  Bodily  reactions  have  relatively  long  latency 
periods,  whereas  reaction  time  for  emotional 
responses  is  often  much  shorter. 

5.  Drugs,  like  adrenalin,  that  produce  bodily 
reactions  do  not  necessarily  produce  emotional 
reactions . 

It  is  these  criticisms  that  led  Cannon  to  develop  his  own 
theory  of  emotions,  known  as  the  Cannon-Bard  theory.  His 
theory  emphasized  the  role  of  sub-cortical  structures,  such 
as  the  thalamus  and  hypothalamus,  in  emotions  [GRIN  78] . 

We  cannot  throw  out  the  James-Lange  theory  altogether. 
Hohmann  reported  evidence  that  showed  the  James-Lange 
theory  to  be  relevant.  In  a  study  conducted  with  patients 
who  had  spinal  cord  lesions,  the  James-Lange  theory  states 
the  patients  would  have  reduced  emotional  experiences  due 
to  the  reduced  number  of  messages  from  the  viscera  (glands 
and  organs)  to  the  brain.  Cannon's  theory  states  such 
lesions  would  have  no  effect  on  the  patients'  emotions. 
Cannon  based  his  statement  on  a  study  conducted  on  dogs 
with  such  lesions,  of  which  he  said  they  responded 
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emotionally.  He  did  point  out  that  he  could  not  tell  if 
the  dogs  still  felt  emotion.  The  human  patients  in  the 
study  reported  feeling  less  emotion  after  the  lesion  than 
before,  but  still  maintained  the  ability  to  experience 
emotion  [HOHM  66] . 

The  effect  of  strain  on  the  body  is  also  an  important 
consideration.  Strain  is  the  term  used  to  describe  the 
impact  of  stress  on  the  body.  The  three  categories  of 
strain  are  physical,  mental,  and  emotional.  Physical 
strain  is  the  actual  physical  load  on  the  body,  i.e. 
lifting  a  heavy  item.  Mental  strain  is  the  mental  effort 
required  by  a  particular  task.  Emotional  strain  is  the 
excess  mental  effort  required  due  to  anxiety  evoking 
concepts  of  the  task,  e.g.  worrying  about  deadlines  [BACK 
00]  .  Each  of  these  different  strains  affects  the  body  in 
different  manners.  Table  4  indicates  how  the  different 
strains  have  been  shown  to  affect  the  physiological 
measure.  The  bolded  measures  are  the  more  common  ones  used 
in  objective  corroborative  experiments  for  measuring 
presence . 
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Table  4.  Psychophysiological  Parameters  of  Different 
Categories  of  Strain  (From:  [BACK  00]). 


Categ 

rory  of  Strain 

Physiological  Measure 

Physical 

Mental 

Emotional 

EEC  alpha  activity  (8-12  Hz) 

ii 

EEC  theta  activity  (4-7  Hz) 

tt 

P3  amplitude 

tt 

P3  latency 

t 

CNV  amplitude 

t 

Heart  rate 

tt 

t 

t 

0.1  Hz  component 

ii 

Respiratory  sinus  arrhythmia 

ii 

Additional  heart  rate 

t 

Respiration  rate 

t 

t 

Finger  pulse  volume 
amplitude 

i 

i 

Systolic  blood  pressure 

tt 

t 

Diastolic  blood  pressure 

t 

t 

EDR  amplitude 

t 

t 

EDR  recovery  time 

t 

Spontaneous  EDR  frequency 

t 

tt 

Eye  blink  rate 

tt 

t 

Saccadic  eye  movements 

t 

Pupillary  diameter 

t 

t 

Electromyogram 

t 

t 

t 

Muscle  tremor 

tt 

Core  temperature 

tt 

Finger  temperature 

t 

Epinephrine 

tt 

t 

Norepinephrine 

tt 

t 

Cortisol 

t 

tt 

Note:  The  variables  are  grouped  according  to  their  respective  physiological  stems. 

"t"  means  that  the  values  of  the  parameter  in  question  increase  with  increasing 
strain,  and  "1"  that  they  decrease.  More  evidence  is  provided  for  double  arrow  than 
for  single  ones. 

Depending  on  the  situation,  the  body  responds  in 
different  manners.  This  is  the  basis  for  the  Three-Arousal 
model  shown  in  figure  4.  Arousal  System  1  is  labeled  the 
affect  arousal  system  and  is  responsible  for  focusing 
attention  and  orienting  responses.  Arousal  System  2,  the 


effort  system,  has  the  ability  to  connect  or  disconnect 

input  and  output.  Arousal  System  3,  activation,  results  in 
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increased  readiness  of  motor  brain  areas  and  is  labeled  the 
preparatory  activation  system.  Affect  arousal  shows  up  in 
phasic  heart  rate  or  EDR  amplitude.  The  effort  system  can 
disconnect  systems  one  and  three  to  prevent  immediate 
action  and  allow  time  for  analysis  [BACK  00] . 


Types  of  Neuronal  System  Examples  for 

Arousal  Predominantly  Involved  Physiological  Measures 


Arousal  1 
“affect 
arousal” 


Arousal  2 
“effort” 


Arousal  3 
“preparatory 
activation” 


Phasic  HR  changes 
Increase  of  EDA 

Decrease  of  alpha 
activity  in  the  EEC 

Decrease  of  HRV 


CNV  in  the  EEC 
Tonic  HR  changes 


Figure  4.  Three  Arousal  Model  for  Engineering 
Psychophysiology  (From:  [BACK  00]). 

The  study  of  emotion  has  taken  four  general  paths: 
emotional  experience,  emotional  arousal,  emotional  action, 
and  emotional  stimulation.  Emotional  experience  refers  to 
the  subjective  reporting  of  states  felt  and  inferred. 
Emotional  arousal  deals  with  changes  within  the  body 
brought  on  by  external  stimuli.  Emotional  actions  are  the 
direct  response  patterns  (e.g.,  attack,  smile,  etc.). 
Finally,  emotional  stimulation  refers  to  the  features  of 
the  environment  that  illicit  one  or  more  of  the  above  [BLAC 
70]  . 
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a.  Physiological  Correlates  of  Emotion 

Many  physiological  changes  can  occur  during 
changes  in  emotional  states.  The  body  controls  its 
physiological  state  through  the  nervous  system.  This 
system  has  two  branches:  the  central  nervous  system  (CNS) 
and  the  peripheral  nervous  system.  The  CNS  includes  the 
brain  and  spinal  cord  while  the  peripheral  system  refers  to 
the  nervous  tissue  external  to  the  brain  and  spinal  cord. 
The  peripheral  system  is  further  divided  into  the  somatic 
nervous  system  and  the  autonomic  nervous  system  (ANS) .  The 
somatic  system  is  concerned  with  muscular  activities  while 
the  ANS  controls  the  body's  visceral  structures.  The  ANS 
is  further  divided  into  the  sympathetic  nervous  system 
(SNS)  and  the  parasympathetic  nervous  system  (PNS) [GRIN 
78]  .  The  SNS  is  dominant  in  situations  requiring  the 
mobilization  of  energy  while  the  PNS  is  dominant  when  the 
body  is  at  rest  [ANDR  80]  .  Table  5  illustrates  which 
portion  of  the  nervous  system  controls  different 
physiological  responses.  See  figure  5  for  a  schematic  of 
the  nervous  system  and  its  major  branches. 


Table  5.  Nervous  System  Mechanisms  in  the  Control  of 
Physiological  Responses  (From:  [ANDR  80]). 


Central  Nervous 

System 

Somatic 

System 

Autonomic  Nervous 

System 

EEG  (ongoing  activity) 

EMG 

Heart  rate  (PNS,  SNS) 

Event -related 

EOG 

Blood  pressure  (PNS,  SNS) 

potentials 

EDA  (SNS  only) 

Pupil  response  (PNS,  SNS) 

Blood  volume  (PNS,  SNS) 
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Nervous  System 

A 

Central  Nervous  Peripheral  Nervous 

System  System 

A 

Somatic  Nervous  Autonomic  Nervous 

System  System 

A 

Parasympathetic  Nervous  Sympathetic  Nervous 

System  System 


Figure  5.  Schematic  Drawing  of  Nervous  System  Showing  the 
Major  Divisions  (From:  [ANDR  80]) . 

The  body  has  many  reactions  that  accompany  emotions. 
The  six  primary  responses  are  heart  rate,  blood  pressure, 
blood  volume,  electrodermal  response,  muscle  potentials, 
and  electroencephalogram  and  related  measures  [GRIN  78]. 
These  measures  can  be  recorded  any  number  of  ways  and  in 
any  combination. 

Heart  rate  is  one  of  the  most  frequently  measured 
responses  and  is  the  most  common  measure  of  heart  activity. 
It  is  usually  expressed  in  beats  per  minute  (bpm) .  The 
normal  rate  for  an  adult  human  heart  averages  around  70bpm, 
although  this  can  fluctuate  from  below  SObpm  to  over  lOObpm 
as  a  result  of  emotions  or  other  factors  [GRIN  78]  .  The 
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beat  represents  the  contracting  of  the  heart  in  order  to 
pump  blood  to  the  rest  of  the  body.  This  beating  is 
controlled  by  both  internal  and  external  mechanisms. 
Internal  mechanisms  consist  of  a  system  of  specialized 
fibers.  External  factors,  such  as  the  nerves  of  the  ANS 
and  CNS,  also  affect  the  rate  at  which  the  heart  beats 
[ANDR  80]  .  Measurement  of  heart  rate  is  typically 
accomplished  by  an  electrocardiogram  (EKG  or  EGG) .  The  EKG 
measures  electrical  potentials  generated  by  the  heart 
muscle  [GROS  67]  .  There  are  several  standard  methods  of 
attaching  electrodes  to  record  EKG  and  are  described  as 
follows  [ANDR  80] : 

Lead  I  -  Attach  electrodes  just  above  the  wrist 
on  both  the  left  and  right  arms  with  the  positive 
lead  on  the  left  arm. 

Lead  II  -  Attach  electrodes  above  the  right  wrist 
and  left  ankle  with  the  ankle  lead  positive. 

Lead  III  -  Attach  electrodes  above  the  left  wrist 
and  left  ankle  with  the  ankle  lead  positive. 

The  lead  placements  described  above  are  adequate  for 
measurement  situations  where  movement  is  minimized  such  as 
sitting,  lying  down,  or  standing  in  one  place.  If  movement 
of  the  participant  is  required,  sternal  or  axillary  leads 
are  preferred.  Sternal  leads  are  placed  over  bones  to 
minimize  movement  artifacts.  Axillary  leads  are  placed  on 
the  muscle  under  the  arm  but  are  not  as  free  from  movement 
artifacts  as  sternal  leads  [ANDR  80] . 

Heart  rate  indicates  physical  as  well  as  mental  load. 
The  different  changes  in  heart  rate  have  different 
meanings.  Tonic  changes  indicate  the  need  to  resupply 
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energy  in  preparation  for  response  while  phasic  changes  can 
be  used  to  determine  stimulus-directed  processing  activity. 
A  decrease  in  finger  volume  pulse  is  a  very  sensitive 
measure  of  defensiveness  and  indicates  both  mental  and 
emotional  strain  [BACK  00] . 

Blood  pressure  is  the  force  exerted  by  the  blood  on 
the  walls  of  the  arteries.  Pressure  increases  as  the  heart 
contracts  and  decreases  as  the  heart  relaxes.  The  maximum 
pressure  that  occurs  when  the  heart  contracts  is  called  the 
systolic  blood  pressure.  The  opposite,  the  minimum 
pressure  that  occurs  when  the  heart  relaxes,  is  the 
diastolic  blood  pressure.  The  difference  between  these  two 
is  called  the  pulse  pressure  [GRIN  78]  .  The  most  common 
method  of  measurement  is  by  an  inflatable  cuff  wrapped 
around  the  upper  arm.  This  cuff  is  called  a 
sphygmomanometer.  Like  most  physiological  signals,  blood 
pressure  fluctuates  continuously,  but  a  measurement  of 
120/80  is  considered  normal  for  young  adults.  Large 
changes  in  blood  pressure,  usually  an  increase,  accompany 
changes  in  emotional  state  [GRIN  78]  . 

Related  to  blood  pressure  is  blood  volume.  Blood 
vessels  can  constrict  and  dilate.  Constriction  is  referred 
to  vasoconstriction  while  vasodilation  refers  to  the 
dilation  of  blood  vessels.  Vasodilation  occurs  when  the 
activity  of  an  organ  increases,  thus  increasing  its  blood 
volume  requirement.  The  vessels  dilate  allowing  more  blood 
to  that  organ.  Along  with  the  increase  of  blood  volume  to 
an  active  organ,  a  similar  decrease  occurs  in  inactive 
organs  by  vasoconstriction  of  the  inactive  organ  vessels 
[GROS  67].  Vasoconstriction  and  vasodilation  of  blood 


32 


vessels  in  the  periphery  of  the  body  is  controlled  by  the 
SNS  and  not  the  PNS;  therefore,  blood  volume  is  a  good 
index  into  SNS  changes  that  occur  due  to  changing  emotional 
states  [GRIN  78] .  This  redistribution  of  blood  can 
sometimes  be  easily  seen  when  a  person  blushes  from 
embarrassment  or  grows  pale  due  to  fear. 

Blood  volume  measurement  is  most  easily  accomplished 
using  a  photoelectric  plethysmograph  as  depicted  in  figure 
6.  It  works  on  the  principle  that  light  is  absorbed  by 
tissue  in  proportion  to  the  amount  of  blood  in  the  tissue 
[GRIN  78]  .  A  beam  of  light  is  directed  into  the  skin  and 
the  resulting  reflection  is  recorded  by  the  photocell.  The 
amount  of  light  reflected  from  the  skin  is  used  as  the 
index  of  blood  volume. 


Figure  6.  Photoelectric  Plethysmograph  (From:  [GRIN  78]) . 

The  fourth  primary  response  to  be  discussed  is 
electrodermal  response  (EDR) .  This  response  has  become  one 
of  the  most  frequently  measured  signals  in  the  field  of 
psychophysiology  [BOUC  92]  .  The  more  common  all 

encompassing  term  is  electrodermal  activity  (EDA) .  The 
basis  of  EDA  is  the  electrical  properties  of  the  skin. 
These  electrical  properties  are  closely  associated  with 
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psychological  processes  such  as  attention  and  emotion  [GRIN 
78] .  The  skin's  resistance,  the  most  commonly  measured 
response,  is  usually  expressed  as  its  reciprocal,  skin 
conductance.  Skin  conductance  is  measured  by  applying  a 
very  small  voltage  across  two  electrodes  placed  on  the 
skin,  usually  the  palm  or  fingers  [GRIN  78]  .  The  activity 
of  the  sweat  glands  is  directly  responsible  for  changes  in 
skin  conductance.  During  emotional  states,  the  skin 
becomes  a  better  conductor  of  electricity.  Skin 
conductance  response  may  indicate  the  amount  of  affective 
arousal  elicited  by  a  situation  or  stimulus  [BACK  00]  . 
However,  electrodermal  activity  can  be  influenced  by  both 
internal  and  external  factors  that  may  cause  variation  in 
the  measured  response.  Studies  show  that  EDA  is  affected 
by  and  dependent  on  ambient  temperature  [BOUC  92]  .  This 
dependency  requires  room  temperature  to  be  held  as  close  to 
constant  as  possible  and  recorded  along  with  other 
experimental  data. 

Internal  influences  of  EDA  are  demographic  in  nature. 
Differences  in  age  groups  have  been  shown  to  exist  but  only 
between  the  third  and  fourth  decades  of  life  [BOUC  92] .  As 
a  body  ages,  the  skin  undergoes  changes  which  commonly 
brings  a  decrease  in  EDA.  A  difference  between  genders  has 
also  been  shown.  Eemale  participants  tend  to  show  a  higher 
tonic  EDA,  while  men  show  more  activity  under  conditions  of 
stimulation.  This  can  be  attributed  to  women  having  more 
sweat  glands  but  men  showing  a  greater  gland  flow  [BOUC 
92] .  Ethnic  differences  also  appear  in  EDA.  This  can  be  a 
result  of  the  decreasing  number  of  active  sweat  glands  with 
increasing  darkness  of  skin.  All  of  these  possible 
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differences  need  to  be  taken  into  account  when  measuring 
EDA. 

All  observable  outward  behavior  is  a  result  of 

muscular  activity,  which  is  measured  as  muscular  potential, 
the  fifth  primary  bodily  response.  Muscles  are  arranged  in 
pairs  of  antagonists,  flexors,  and  extensors  [GROS  67]  . 
Muscles  are  at  the  lowest  level  of  tension  when  in  a 
relaxed  state  (i.e.  sleep)  and  increase  as  the  body 

prepares  for  action.  This  tension  is  a  frequent  correlate 
of  emotion  [GRIN  78]  .  Measurement  of  muscular  potential 
can  be  generally  accomplished  two  ways,  internally  or 
externally.  Internally  requires  the  insertion  of  a  needle 
into  the  muscle  to  record  the  electrical  signal  of  the 
muscle  and  is  commonly  called  an  electromyogram  (EMG) . 
Externally  does  not  require  the  piercing  of  the  skin. 

Instead,  electrodes  are  placed  on  the  surface  of  the  skin 
over  the  muscle  of  interest.  This  method  is  usually  called 
either  a  surface  EMG  or  muscle  action  potential  (MAP) . 
Unlike  the  other  responses  discussed  so  far,  muscle  fibers 
are  innervated  by  the  CNS  and  not  the  ANS  [GRIN  78] . 

The  last  of  the  primary  bodily  responses  to  be 
discussed  is  the  electroencephalogram  (EEG)  and  related 
measures.  The  EEG  measures  electrical  potentials 

continuously  emitted  by  the  brain.  Electrodes  placed  on 
the  scalp  accomplish  measurement  of  the  EEG.  The  EEG  is 
made  up  of  many  frequencies.  Some  of  these  frequencies 
show  up  more  often  than  others.  These  frequencies  are 
labeled  delta  waves  (less  than  4Hz),  theta  waves  (4-7Hz) , 
alpha  waves  (8-13Hz),  and  beta  waves  (greater  than  13Hz) . 
Alpha  waves  are  the  most  common  in  the  normal  awake  adult 
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and  are  associated  with  relaxation.  Beta  waves,  the  next 
most  common,  are  associated  with  excited  states.  Theta 
waves  are  not  normal  for  adults  but  are  the  most  common  in 
children.  Delta  waves  are  only  common  in  certain  stages  of 
sleep  [GRIN  78]  .  Since  the  EEG  is  a  measure  of  brain 
activity,  it  reflects  the  activity  of  the  CNS  and  not  the 
ANS .  A  summary  of  the  six  primary  responses  is  shown  in 
table  6. 


Table  6.  Summary  of  Six  Major  Responses  (Erom:  [GRIN  78]) . 


Response 

Primary 

Unit  of 

SNS 

PNS 

Recording 

System 

Organ 

Measurement 

Effect 

Effect 

Technique 

Heart  rate  (HR) 

Heart 

Beats  per 
minute  (BPM) 

Increase 

Decrease 

Electrodes 
(placed  on  both 
sides  of  the 
heart ) 

Blood  pressure  (SBP, 
DBP,  PP) 

Heart 

and 

blood 

vessels 

Millimeters 
of  mercury 
pressure  (mm 
Hg) 

Increase 

Decrease 

Sphygmomanometer 
(with  pressure 
cuff  wrapped 
around  the  upper 
arm) 

Peripheral  blood 
volume  (BV,  PV) 

Blood 

vessels 

Relative 
change  in 
millimeters 
(mm) 

Decrease® 

None 

Photoelectric 

plethysmography 

Electrodermal  skin 
conductance  (SCL, 

SCR) 

Sweat 

glands 

Micromhos 

Increase 

None 

Electrical 
voltage  (applied 

across 

electrodes  on 
the  skin) 

Muscle  potential 
(EMG,  MAP) 

Skeletal 

muscles 

Microvolts 

or 

millivolts 

None 

None 

Electrodes 
(placed  on  the 
skin  above  the 
muscle) 

Electroencephalogram 
(EEG,  CER,  CNV) 

Brain 

Microvolts 

None 

None 

Electrodes 
(placed  on  the 
scalp) 

■^This  table  lists  only  SNS  and  PNS  effects;  it  is  important  to  remember  that 
determination  from  parts  of  the  CNS  are  also  important. 


to 


There  are  five  other  bodily  responses  that 
emotions  but  are  measured  less  frequently. 


are  related 
These  are 
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respiration,  temperature,  salivation,  pupil  size  and 
gastric  motility. 

Respiration  refers  to  breathing  measures.  The  most 
common  measures  are  respiration  rate  (breaths  per  minute) , 
respiration  period  (time  between  breaths) ,  and  respiration 
volume  (amount  of  air  inhaled/exhaled).  Respiration  is 
controlled  by  both  the  CNS  and  ANS  [GRIN  78] . 

Temperature  is  the  measurement  of  the  body' s  heat  and 
is  of  two  types,  general  or  local.  General  temperature  is 
a  measurement  of  the  body' s  overall  thermal  output  and  is 
obtained  with  either  an  oral  or  rectal  thermometer.  Local 
temperature  is  determined  by  attaching  a  thermistor  or 
thermocouples  to  a  particular  body  part,  usually  the  hands 
or  fingers.  It  is  mostly  determined  by  the  amount  of  blood 
in  that  part  of  the  body  and  is,  therefore,  controlled 
indirectly  by  the  SNS . 

Salivation  refers  to  the  amount  of  and  composition  of 
secretions.  Salivary  glands  are  activated/deact ivated  by 
the  PNS  and  SNS.  Output  is  activated  by  the  PNS  and 
deactivated  by  the  SNS  [GRIN  78] . 

Pupil  size,  another  infrequent  measurement,  can  range 
from  1.5mm  to  more  than  9mm.  The  PNS  controls  constriction 
while  the  SNS  controls  dilation.  The  last  bodily  response 
to  be  discussed  is  gastric  motility.  This  is  usually 
measured  by  swallowing  a  monitoring  device.  The  device  can 
be  either  a  magnet  or  a  transmitter.  The  stomach  and 
intestines  are  controlled  by  both  the  PNS  and  SNS  [GRIN 
78]  . 
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b.  Physiological  Patterns  of  Emotion 

William  James'  theory  of  emotion  theorized  that  a 


body' s  physiological  responses  could  be  used  to  recognize 
emotion.  Critics  of  this  argue  that  only  certain  aspects 
of  emotion  can  be  determined  from  physiology,  but 
researchers  have  identified  physiological  signals  that 
differentiate  emotional  states  [HEAL  00] . 

There  are  two  basic  patterns  used  in  emotional 
classification:  arousal-valence  and  name  classification. 
The  arousal -valence  classification  is  illustrated  in  figure 
7  . 


The  arousal  axis  is  characterized  by  a  continuous  response 
from  calm  and  peaceful  to  active  and  excited.  The  valence 
axis  ranges  from  negative  to  positive  [VYZA  98]  .  The  two 
dimensions  of  arousal  and  valence  account  for  most  of  the 
variance  in  the  measurement  of  emotions  [DETE  98] . 

Name  classification  is  somewhat  more  complex.  Like 
most  other  situations  where  the  true  definition  of  a 
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concept  is  nebulous,  emotion  theorists  have  proposed  sets 
of  basic  emotions  as  illustrated  in  table  7.  As  the  table 
shows,  one  theorist's  idea  of  the  basic  emotions  is  not 
necessarily  another's.  However,  the  table  also  shows  that 
some  emotions  are  members  of  more  than  one  set  [ORTO  88]  . 
A  difficulty  of  using  name  classification  is  the  lack  of 
clarity  between  the  emotion  categories.  There  is  no  way  to 
determine  the  similarity  or  difference  of  emotions.  This 
problem  also  makes  it  difficult  to  differentiate  between 
examples  of  the  same  emotion  [HEAL  00] . 


Table  7.  Summary  of  Emotion  Sets  Proposed  by  different 
Theorists  (Erom:  [ORTO  88]). 


Theorist 

Emotion  Set 

James 

Rage,  fear,  grief,  love 

Ekman 

Anger,  fear,  sadness,  enjoyment, 
disgust,  (surprise) 

Clynes 

Anger,  hate,  grief,  joy,  love,  romantic 
love,  reverence,  no  emotion 

Panskepp 

Rage,  fear,  panic,  expectancy 

Plutchik 

Anger,  fear,  anticipation,  sadness,  joy, 
acceptance,  disgust,  surprise 

Izard 

Anger,  fear,  distress,  joy,  surprise, 
interest,  disgust,  contempt,  guilt, 
shame 

Erl jda 

Anger,  fear,  distress,  joy,  surprise, 
aversion,  contempt,  pride,  shame,  desire 

The  role  of  physiology  in  assessing  emotion  has  been 
highly  debated.  Much  research  has  been  done  in  trying  to 
find  physiological  correlates  which  can  be  used  to  detect 
emotional  state.  While  there  is  not  a  direct  link  between 
specific  physiological  response  and  emotional  state,  there 
may  be  a  link  between  groups  of  physiological  responses  and 
emotional  states.  Cacioppo  and  Tassinary  proposed  a  new 


model  that  uses  mathematical  combinations  of  features  to 
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better  distinguish  emotional  states  [CACI  90] .  They 
theorize  that  if  features  are  combined,  emotional  states 
may  be  more  distinguishable.  Figure  8  depicts  a 

theoretical  mapping.  It  can  be  seen  from  figure  8  that  the 
emotional  states  may  be  hard  to  distinguish  if  the 
physiological  signals  are  considered  separately.  However, 
if  the  physiological  signals  are  combined  in  different 
profiles,  the  emotional  states  may  have  a  one  to  one 
mapping  with  physiological  profiles. 


PSYCHOLOGICAL 

DOMAIN 


d) 


PHYSIOLOGICAL 

DOMAIN 


RGlaxation 

Orienting 

Startle 

Defense 


SCR 

Decelerating  HR 
Accelerating  HR 


Relaxation 

Orienting 

Startle 

Defense 


No  SCR  and  Decelerating  HR 

SCR  and  Decelerating  HR 
SCR  and  Accelerating  HR 


Relaxation 

Orienting 

Startle 

Defense 


No  SCR  and  Decelerating  HR 
SCR  and  Decelerating  HR 
Abrupt  accelerating  HR 

Lingering  accelerating  HR 


Figure  8.  Combining  Physiological  Signals  to  Distinguish 
Emotional  State  (From:  [CACI  90]). 


c.  Previous  Research  in  the  Physiological 
Aspects  of  Emotions 
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In  the  world  of  psychophysiology,  much  research  has 
gone  into  trying  to  determine  a  person' s  emotional  state  by 
measuring  his  physiological  responses.  While  a  direct 
mapping  between  emotion  and  physiological  response  has  not 
been  found,  specific  relationships  have  been  found  between 
the  two  most  commonly  cited  dimensions  of  emotion  -  arousal 
and  valence  [DETE  98]  .  Eor  valence,  heart  rate  and  facial 
muscle  activity  have  been  found  to  be  good  indicators  while 
the  frequency  and  amplitude  of  skin  conductance  response  is 
a  good  indicator  of  arousal  [SIMO  99] [AX  64] . 

More  recently,  research  in  the  area  of  affective 
computing  has  taken  on  the  task  of  using  the  body' s 
physiological  responses  to  determine  the  emotional  state  of 
the  user.  Affective  computing  has  been  defined  as 
"computing  that  relates  to,  arises  from,  or  deliberately 
influences  emotions"  [EERN  97].  The  primary  motivation 
behind  this  line  of  research  is  to  develop  computer 
interfaces  that  can  actually  determine  the  user' s  emotional 
state  and  respond  accordingly. 

Initial  research  into  the  area  of  affective  computing 
revolved  around  recognition  of  emotional  state.  Two 
experiments  conducted  in  emotion  recognition  centered  on 
intentionally  expressed  emotions.  The  experiments  recorded 
four  physiological  signals  (electromyogram,  skin 
conductivity,  blood  volume  pulse,  and  respiration) ,  which 
were  later  processed  for  emotion  recognition.  Both 
resulted  in  emotion  recognition  with  greater  than  chance 
probabilities  [HEAL  98]  [VYZA  98]  .  Particularly,  anger  was 
easily  differentiated  from  peaceful  emotions  and  high 
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arousal  states  were  distinguishable  from  low  arousal  states 
[HEAL  98]  . 

A  similar  study  attempted  to  determine  when  a  user  was 
frustrated,  particularly  the  frustration  experienced  by  a 
computer  user,  using  physiological  signals.  The 
physiological  signals  measured  were  GSR,  BVP,  and  EMG. 
These  three  signals  have  been  shown  to  covary  with 
increased  frustration  and  anxiety  [RISE  98]  .  Blood  Volume 
Pulse  has  been  identified  as  a  measure  of  anxiety  in 
response  to  a  threat  [SMIT  84]  .  Physiological  signals,  as 
measured,  were  able  to  differentiate  between  frustration 
states  and  non-frustration  states  using  Hidden  Markov 
Models  [RISE  98]  . 

Another  more  recent  application  of  physiological 
monitoring  is  the  SmartCar  experiment  by  MIT.  In  this 
experiment,  four  physiological  signals,  EKG,  EMG, 
respiration,  and  GSR,  were  used  to  detect  driver  stress  in 
real  life  situations.  The  signals  were  chosen  due  to  their 
recognition  as  useful  indicators  for  assessing  arousal  and 
stress.  Using  combinations  of  features  from  multiple 
physiological  signals  improved  the  detection  of  stress  over 
single  features  [HEAL  00] . 

4 .  Linking  Objective  and  Subjective  Measures  of 
Presence 

Presence  is  commonly  defined  as  the  subjective  sense 
of  "being  there"  in  a  mediated  environment.  Due  to  its 
subjective  nature,  the  most  common  method  of  measuring 
presence  is  through  post-test  questionnaires.  Measuring 
presence  in  this  manner  makes  the  measure  dependent  on 
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memory  [DILL  00]  .  An  aspect  of  presence  that  is  usually 
disregarded  in  current  definitions  is  the  emotional  aspect. 
Emotions  are  considered  essential  to  how  people  experience 
the  world  around  them  [HUAN  99]  .  In  fact,  emotions  are  an 
every  day  part  of  life.  How  people  experience  a  virtual 
world  will  also  involve  some  aspect  of  their  emotions. 

The  two  most  commonly  cited  dimensions  of  emotion  are 
arousal  and  valence.  As  stated  earlier,  arousal  is  usually 
considered  to  range  from  excited  and  alert  to  calm  and 
peaceful  while  valence  is  characterized  as  pleasant  to 
unpleasant  [DETE  98]  .  Arousal  can  be  further  conceived  as 
a  "drive  state  or  a  non-specific  energizer  of  behavior" 
[DILL  00]  .  It  is  usually  linked  to  the  intensity  of  the 
experience  and  not  the  quality.  Arousal  is  associated  with 
physiological  activity  through  the  autonomic  nervous  system 
(ANS)  [DILL  00] [GRIN  78] . 

Eor  this  research,  the  measurements  of  electrodermal 
activity  (EDA) ,  heart  rate  (HR) ,  and  skin  temperature  will 
be  recorded.  Electrodermal  activity  has  been  shown  to  have 
a  specific  link  to  arousal  and  HR  is  linked  to  hedonic 
valence  [DETE  98] .  As  the  intensity  of  the  stimulus 
increases,  so  does  the  body's  EDA.  Electrodermal  activity 
has  also  been  shown  to  occur  on  the  presentation  of 
unexpected  stimuli,  which  is  commonly  referred  to  as  the 
orienting  response.  Meehan  found  that  EDA  has  a  high 
correlation  with  subjective  presence  ratings  [MEEH  00]. 
Heart  rate,  on  the  other  hand,  is  affected  by  the  intensity 
of  the  emotional  stimulus.  Positive  emotions  involve 
greater  heart  activity  than  negative  emotions  [DILL  00]  . 
Heart  rate  is  also  responsive  to  unexpected  stimuli.  Heart 
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rate  decelerations  are  associated  with  the  orienting 
response  while  accelerations  are  associated  with  defensive 
responses  [GRAH  92] .  Skin  temperature  is  indicative  of  the 
amount  of  blood  in  the  area.  This  is  directly  controlled 
by  the  vasoconstriction  or  vasodilation  of  the  blood 
vessels,  which  is  controlled  indirectly  by  the  sympathetic 
division  of  the  autonomic  nervous  system  (ANS)  [GRIN  78]  . 
Skin  temperature  has  been  shown  to  fall  during  emotional 
reactions  [GROS  67]  and  is  believed  to  be  a  valid  measure 
for  fear  response  [MEEH  00] . 

The  subjective  sensation  of  "being  there"  in  a 
mediated  environment  is  the  basis  for  the  definition  of 
presence.  But  what  does  it  mean  to  "be  there"?  Being 
physically  present  in  an  environment  is  not  sufficient. 
How  many  times  have  you  sat  in  a  classroom  but  did  not  hear 
a  word  the  instructor  said?  Or  have  you  ever  driven  your 
car  down  a  specific  section  of  road  that  you  are  intimately 
familiar  with  only  to  realize  once  you  reach  your 
destination  that  you  cannot  remember  how  you  got  there? 
What  is  lacking  is  the  psychological  state  of  the 
participant.  A  person's  psychological  state  is  partly 

made  up  of  his  emotional  state  and  the  perception  of  his 
body's  responses.  Therefore,  by  measuring  the  body's 
physiological  responses  when  exposed  to  a  virtual 
environment,  the  user's  emotional  state  is  indirectly 
measured.  This  measurement  of  emotional  state  might  yield 
a  valid  objective  measure  of  presence. 
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III.  METHOD 


A.  EXPERIMENT  DESIGN 

This  thesis  will  assess  the  impact  of  sound  delivery 
method  on  a  user's  sense  of  presence.  To  do  this,  each 
participant  will  be  placed  in  a  VE  and  perform  various 
tasks  associated  with  the  VE .  Sound  delivery  will  be  the 
independent  variable  of  this  study.  Physiological 
responses  will  be  the  primary  dependent  measures  collected. 
These  responses  provide  a  good  indication  of  the 
participant's  emotional  state  during  exposure  to  stimuli 
(see  ch .  II,  sec.  3) .  The  next  step  is  to  make  the  logical 
link  between  emotional  state  and  presence  in  a  virtual 
environment  (see  ch .  II,  sec.  4)  .  An  additional  effort 
will  be  made  to  investigate  correlation  between 
physiological  responses  and  subjective  questionnaire 
scores.  This  could  establish  a  potential  link  between 
objective  and  subjective  measures  of  presence.  Eigure  9 
outlines  this  process. 
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[Link  of  Physiological  I 

ResponsetoPresence  I 

Figure  9.  Experiment  Design  Logic. 

B.  VIRTUAL  ENVIRONMENT  SELECTION 

The  virtual  environment  required  for  this  research  had 
to  meet  two  main  criteria:  excellent  graphics  and  state  of 
the  art  sound  quality.  Several  first  person  shooter  style 
games  were  evaluated  but  Medal  of  Honor:  Allied  Assault™ 

by  EA  Games™  (MOHAA)  was  the  one  selected.  Medal  of  Honor: 
Allied  Assault™  by  EA  Games™  was  preferred  primarily  due  to 
its  superb  graphics  and  impressive  sound  quality.  The  game 
is  based  on  the  European  campaigns  of  World  War  II.  While 
the  scenarios  have  a  historical  perspective,  it  is  much 
closer  to  reality  than  any  of  the  other  first  person 
shooter  games  evaluated.  Another  selling  point  for  this 
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game  engine  was  its  uniqueness.  There  was  little  concern 
about  negative  effects  on  presence  due  to  previous  exposure 
to  the  environment  since  MOHAA  had  recently  been  released 
(January  2002)  . 

The  MOHAA  scenario  chosen  was  the  Omaha  Beach  Landing 
of  the  Normandy  Invasion.  This  scenario  provided  a  highly 
stimulating  environment  mentally,  visually,  and  aurally. 
The  participants  started  game  play  with  full  health  at  the 
shingle  (see  figures  10  and  11)  .  The  objective  of  the 
scenario  was  to  clear  out  the  bunkers  defending  the  beach 
(see  Appendix  C)  .  The  reason  for  choosing  the  shingle 
starting  point  was  the  actual  landing  on  the  beach, 
although  intense,  appeared  to  be  too  difficult.  The 
concern  was  that  this  difficulty  would  induce  frustration 
in  the  novice  player,  thereby  introducing  inadvertent 
changes  in  physiological  responses.  Frustration  would  have 
an  impact  on  the  user' s  physiological  responses  and  could 
obscure  the  physiological  changes  due  to  presence. 


Figure  11.  Players  View  of  Starting  Point  (From: 

www.gamespot.com) . 
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C.  EXPERIMENTAL  DESIGN 

The  experiment  was  a  between  participant  design  with 
the  sound  delivery  method  being  the  independent  variable. 
There  were  four  delivery  methods:  No  sound.  Speakers, 
Headphones,  and  Headphones  with  subwoofer.  All  sound 
treatments  had  to  be  delivered  at  the  same  intensity  at  the 
ear  of  the  listener.  In  order  to  do  this,  appropriate 
sound  settings  by  treatment  had  to  be  determined.  This  was 
accomplished  using  a  CEL  Instruments  Digital  Sound  Survey 
Meter  (see  Appendix  F  for  specifications) .  The  intensity 
was  set  at  approximately  90  dB  for  each  treatment.  The 
bass  was  also  adjusted  to  ensure  that  both  subwoofer 
treatments  received  the  same  intensity.  To  accomplish 
this,  the  bass  volume  was  measured  for  the  speaker 
treatment  and  then  the  appropriate  bass  level  was  matched 
for  the  headphones  with  subwoofer  treatment.  The  intensity 
settings  by  treatment  are  located  in  Appendix  B. 

A  between  participant  design  was  chosen  to  minimize 
the  effect  of  repeated  exposure  to  the  environment.  Meehan 
showed  that  repeated  exposure  to  a  virtual  environment 
decreases  the  user's  sense  of  presence  [MEEH  00] . 

The  dependent  measures  in  this  experiment  were  the 
participant's  physiological  responses  and  their  responses 
to  subjective  questionnaires.  Three  questionnaires  were 
used  in  the  experiment,  an  immersive  tendencies 
questionnaire  and  two  presence  questionnaires  (see  Appendix 
E)  . 

Immersive  Tendencies  Questionnaire:  The  immersive 
tendencies  questionnaire  (ITQ)  consists  of  eighteen 


questions  designed  to  identify  real-world  behaviors  and 
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tendencies  that  may  predict  an  individual's  potential  to 
experience  presence.  The  questions  are  rated  on  a  seven 
point  Likert-scale  ranging  from  one  to  seven.  An 
individual's  ITQ  score  is  the  sum  of  these  values  [SING 
96] .  An  individual  with  a  higher  ITQ  score  is  expected  to 
also  score  high  on  the  presence  questionnaires.  The  ITQ 
has  been  used  by  other  researchers  in  the  study  of  presence 
[DARK  99]  [LAWS  98]  . 

Presence  Questionnaires:  After  exposure  to  the 
environment,  the  participants  answered  a  combination  of  two 
questionnaires  dealing  with  their  sense  of  presence  within 
the  environment.  The  questions  on  the  post  questionnaire 
came  from  Witmer  and  Singer' s  Presence  Questionnaire 
(PQ) [SING  96]  and  Slater's  six  questions  on  presence  [SLAT 
95]  .  Both  of  these  questionnaires  were  rated  on  a  seven- 
point  Likert-scale  similar  to  the  ITQ.  Witmer  and  Singer's 
questionnaire  consisted  of  twenty-four  questions  with 
presence  scores  being  determined  by  summing  up  the  ratings 
of  these  twenty-four  questions.  Slater's  questionnaire 
consisted  of  six  questions  with  the  presence  score 
resulting  from  summing  up  the  number  of  high  answers  (6's 
or  7's)  .  These  two  questionnaires  were  used  in  an  effort 
to  investigate  the  correlation  between  the  questionnaires 
since  the  authors  have  a  difference  of  opinion  on  the 
definition  of  presence. 

The  physiological  responses  measured  were  heart  rate, 
electrodermal  activity  (EDA)  and  temperature.  These  were 
chosen  as  outlined  in  the  background  section.  EDA  is  an 
indicator  of  arousal,  heart  rate  follows  hedonic  valence. 


50 


and  temperature  has  been  shown  to  be  an  indicator  of  fear 
response . 

D .  EQUIPMENT 

Three  computers  were  used  in  the  experiment:  two  Dell™ 
Dimension  8100s  and  one  Alienware™  computer  (see  Appendix 
F)  .  One  Dell  contained  the  questionnaires  while  the  other 
was  used  to  record  the  participant's  physiological 
responses.  The  environment  was  run  on  the  Alienware™ 
computer  using  a  Creative  Labs  Audigy  Sound  Card  and 
presenting  graphics  on  an  NVIDIA  GeforeS  graphics  card. 

The  questionnaire  computer  made  use  of  a  program 
called  MediaLab™  by  Empirisoft™.  MediaLab™  allows  for  easy 
generation  and  recording  of  questionnaire  data.  Once  the 
participant  answered  the  computer-based  questionnaire, 
their  answers  were  recorded  and  saved  in  a  spreadsheet 
format  for  later  analysis. 

Physiological  responses  were  monitored  and  recorded 
using  Thought  Technology's™  BioGraph™  2.1  software  and 
ProComp+™  distribution  block  (see  Appendix  F)  .  ProComp+™ 
allows  simultaneous  recording  of  up  to  eight  different 
physiological  signals.  BioGraph™  2.1  also  includes  a 
simple  statistical  package  that  allows  for  easy  calculation 
of  the  mean  and  variance  for  each  recorded  signal. 

The  headphone  used  was  a  Sennheiser™  model  HD570  with 

an  open  design  (see  Appendix  F)  .  The  same  headphone  was 

used  for  both  the  headphones  and  headphones  with  subwoofer 

treatments.  The  surround  sound  speaker  configuration 

consisted  of  five  Genelec™  1031A  active  speakers  with  a 

single  Genelec™  1094A  active  400  watt  subwoofer  system  (see 
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Appendix  F) .  The  1094A  subwoofer  was  also  used  in 
conjunction  with  the  headphones  for  the  headphones  with 
subwoofer  treatment. 


E .  PARTICIPANTS 

A  total  of  eighty  (80)  participants  were  used  in  the 
study  with  a  breakdown  of  seventy-six  (76)  males  and  four 
(4)  females.  The  age  breakdown  of  the  participants  is 
illustrated  in  figure  12.  All  participants  were  affiliated 
with  the  Naval  Postgraduate  School  either  as  students, 
instructors,  or  employees.  Prior  experience  with  VEs 
varied  among  all  subjects  as  well  as  first  person  shooter 
gaming  experience.  Gaming  experience  breakdown,  as 
reported  by  the  participant,  is  shown  in  figure  13. 
Participants  were  assigned  to  treatment  groups  at  random. 
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Figure  12.  Age  Breakdown  by  Condition. 
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rience  by  Condition. 


F .  PROCEDURES 

Step  One:  Upon  entering  the  lab,  the  participant  was 
handed  consent  forms  and  asked  to  read  and  sign  all  three 
forms  (Appendix  D)  .  The  researchers  then  directed  the 
participant  into  the  Multimedia  Lab  and  assigned  the 
participant  an  identification  number  and  treatment. 

Step  Two:  Once  seated  in  the  Multimedia  Lab,  the 
participant  was  directed  to  follow  the  commands  given  by 
the  "drill  instructor"  and  complete  the  Basic  Training 
scenario  in  Medal  of  Honor:  Allied  Assault™.  This  step 
allowed  the  participant  to  become  familiar  with  the  keys 
used  to  maneuver  within  the  virtual  environment. 
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step  Three:  Upon  completion  of  basic  training,  the 

participant  was  then  directed  to  the  questionnaire  computer 
and  asked  to  complete  the  pre-questionnaire  (Immersive 
Tendencies  Questionnaire) . 

Step  Four:  While  the  participant  was  answering  the 

ITQ,  the  researchers  set  up  the  virtual  environment  in 
accordance  with  the  participant's  assigned  treatment  (see 
step  VII,  Appendix  B) . 

Step  Five:  After  completing  the  ITQ,  the  participant 

was  then  directed  back  into  the  Multimedia  Lab. 

Physiological  sensors  were  attached  to  the  participant  in 
accordance  with  step  VI  of  Appendix  B.  Once  all  sensors 

were  attached,  the  participant  was  handed  the  Intel  Brief 
(Appendix  C)  and  asked  to  sit  still  and  quiet  while  his 

two-minute  baseline  physiological  responses  were  recorded. 
These  recordings  were  saved  under  the  filename  of 

subxxbase,  where  xx  is  the  participant's  ID  number. 

Step  Six:  When  the  participant  indicated  he  was  ready 
to  continue,  the  video  camera  and  physiological  recording 
were  started.  The  participant  was  then  told  to  start  the 
mission.  After  ten-minutes  of  physiological  responses  were 
recorded,  the  participant  was  told  to  stop.  The 
physiological  sensors  were  removed  and  the  participant  was 
directed  back  to  the  questionnaire  computer  to  complete  the 
post-test  questionnaire. 


54 


IV.  J^ALYSIS  AND  DISCUSSION 


A.  INTRODUCTION 

The  analysis  was  done  in  two  parts.  First,  the 
physiological  responses  were  analyzed  to  determine  the 
effect  of  sound  delivery  on  the  user's  sense  of  presence. 
Next,  the  physiological  responses  were  compared  to  the 
questionnaire  data  to  investigate  correlation. 

For  the  following  analyses,  an  a  level  of  0.10  was 
chosen.  This  less  conservative  level  of  significance  was 
selected  due  to  the  variability  in  the  individual 
differences  of  the  participants,  both  in  their 
physiological  and  subjective  responses. 

B .  PRIMARY  RESULTS 

1 .  Primary  Hypothesis 

The  method  of  sound  delivery  used  in  a  virtual 
environment  will  have  no  significant  effect  on  the  user's 
sense  of  presence. 

Baseline  vs.  Baseline:  The  first  step  in  the  analysis 
of  the  physiological  responses  was  to  determine  if  any 
physiological  difference  existed  between  the  participants 
for  each  condition.  Comparison  of  the  baseline  readings 
(heart  rate,  electrodermal  activity,  and  temperature)  by 
condition  revealed  no  significant  difference  (see  table  8)  . 
This  was  desired  since  any  difference  in  baseline  readings 
between  conditions  would  obscure  any  further  results. 
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Table  8.  ANOVA  of  Baseline  Physiological  Response  by 

Condition . 


Physiological  Response 

df 

F-statistic 

P 

Electrodermal  Activity 

(3,  76) 

0.768 

0.515 

Heart  rate 

(3,  76) 

1.662 

0.182 

Temperature 

(3,  76) 

0.989 

0.403 

Baseline  vs.  Condition:  The  next  step  was  to  compare 
the  physiological  responses  by  condition  to  their 
respective  baseline  readings.  The  results  were  mixed  (see 
table  9)  . 

EDA  Results:  Visual  stimuli  alone  had  no  significant 
effect  on  EDA  (no  sound  condition)  ,  but  the  addition  of 
sound  (speakers,  headphones,  and  headphones  with  subwoofer) 
to  the  virtual  environment  created  a  significant  change  in 
EDA.  Electrodermal  activity  is  an  indicator  of  arousal 
[DETE  98]  .  Thus,  audio  increases  arousal  and  according  to 
our  operational  definition  of  presence,  leads  to  a  change 
in  a  user's  sense  of  presence. 

Heart  Rate  Results:  Heart  rate  was  not  significantly 
affected  by  any  of  the  conditions. 

Temperature  Results:  The  only  sound  condition  to  show 
a  significant  effect  on  temperature  was  the  speaker 
condition.  Temperature  has  been  shown  to  fall  during 
emotional  reactions  [GROS  67]  and  is  considered  a  valid 
measure  of  fear  response  [MEEH  00] . 

In  summary,  comparing  the  average  baseline  responses 
to  the  average  responses  during  game  play  shows  that  sound 
affects  the  user's  emotional  state.  ^ecifically,  EDA  is 
affected  by  the  inclusion  of  sound  to  the  environment 
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regardless  of  the  delivery  method.  Temperature  is 

significantly  affected  by  the  speaker  condition  only. 


Table  9.  Comparison  of  Baseline  and  Condition 
Physiological  Responses. 


Condition  EDA  vs  Baseline  EDA 

df 

F-statistic 

P 

No  sound 

(1, 

38) 

1 . 747 

0.194 

Speakers 

(1, 

38) 

2.953 

0.094 

Headphones 

(1, 

38) 

4.990 

0.031 

Headphones  w/  subwoofer 

(1, 

38) 

3.821 

0.058 

Condition  Heart  Rate  vs  Baseline 

Heart  Rate 

df 

F-statistic 

E 

No  sound 

(1, 

38) 

0 . 729 

0.398 

Speakers 

(1, 

38) 

1 . 770 

0 .191 

Headphones 

(1, 

38) 

0.865 

0.358 

Headphones  w/  subwoofer 

(1, 

38) 

1.601 

0.214 

Condition  Temp,  vs  Baseline  Temp. 

df 

F-statistic 

P 

No  sound 

(1, 

38) 

0.062 

0.805 

Speakers 

(1, 

38) 

3.672 

0.063 

Headphones 

(1, 

38) 

0.530 

0 . 471 

Headphones  w/  subwoofer 

(1, 

38) 

1.217 

0.277 

Condition  vs.  Condition:  The  next  analysis  compared 
the  changes  in  the  physiological  responses  between  the 
different  conditions. 

EDA  Results:  No  significant  difference  between 
conditions . 

Heart  Rate  Results:  No  significant  difference  between 
conditions . 

Temperature  Results:  Temperature  was  the  only 
response  to  show  a  significant  difference  between 
conditions.  This  significance  is  shown  between  no  sound 
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and  any  sound  condition  and  between  speakers  and  both 
headphone  conditions.  No  significance  was  shown  between 
headphones  and  headphones  with  subwoofer  (see  tables  10  and 
11)  .  A  look  at  the  average  change  in  temperature  by 
condition  revealed  speakers  had  the  greatest  effect  on 
temperature  with  an  average  decline  of  3.36  degrees 
Fahrenheit  (see  table  12)  .  Again,  temperature  drops  have 
been  linked  to  changes  in  emotional  state.  This  also 
coincides  with  the  findings  tabulated  in  table  9. 

A  similar  analysis  with  the  exclusion  of  suspected 
contaminators  (pilot  study  participants,  previous  Medal  of 
Honor  players,  and  any  identified  outliers)  also  resulted 
in  only  a  significant  effect  in  temperature. 


Table  10.  Comparison  of  Change  in  Physiological  Responses 
_ by  Condition. _ 


Change  in  Physiological  Response  by  Condition 

df 

F  statistic 

E 

Electrodermal  Activity  (EDA) 

(3,  76) 

0.755 

0.523 

Heart  Rate  (HR) 

(3,  76) 

0.306 

0.821 

Temperature 

(3,  76) 

1 .156 

0.0001 

Table  11.  Comparison  of  Change 

in  Temperature. 

Condition  versus  Condition 

df 

t  Critical 

(two-tailed) 

E 

No  sound  vs  Speakers 

19 

1.729 

0.00001 

No  sound  vs  Headphones 

19 

1.729 

0.060 

No  sound  vs  Headphones  w/subwoofer 

19 

1.729 

0.014 

Speakers  vs  Headphones 

19 

1.729 

0.003 

Speakers  vs  Headphones  w/subwoofer 

19 

1.729 

0.056 

Headphones  vs  Headphones  w/subwoofer 

19 

1.729 

0.221 
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Table  12.  Average  Change  in  Temperature  by  Condition. 


No  sound 

Speakers 

Headphones 

Headphones  w/ 
Subwoofer 

o 

o 

1 

-3.36 

-1.70 

-2.35 

2 .  Secondary  Hypothesis 

Physiological  responses  of  heart  rate,  elect rodermal 
activity,  and  temperature  can  be  used  as  objective  measures 
of  presence. 

Although  the  physiological  results  met  our  criteria 
for  measuring  presence  using  our  operational  definition  of 
presence,  we  wanted  to  determine  if  these  physiological 
measures  correlated  with  questionnaires  commonly  used  to 
measure  presence.  The  analytical  tool  chosen  for  this 
procedure  was  the  Spearman  Rank  Correlation  due  to  its 
applicability  for  non-linear  relationships  [MEEH  00]. 

Questionnaire  Analysis:  The  first  step  in  the 

questionnaire  analysis  was  to  look  at  the  relationship 
between  the  scores  on  the  Immersive  Tendencies 
Questionnaire  (ITQ)  and  the  Presence  Questionnaire  (PQ)  . 
The  next  step  was  to  break  apart  the  post  questionnaire 
into  the  Witmer  and  Singer  PQ  and  the  Slater  questions  (SQ) 
and  analyze  this  data  to  see  if  a  correlation  between 
questionnaires  exists. 

A  Spearman  rank  correlation  of  the  ITQ  score  and  PQ 
score  reveals  rg  =  0.46,  P  =  0.0.  The  results  indicate  a 
high  overall  correlation  between  the  two  questionnaires 
exists.  This  is  not  surprising  since  the  purpose  of  the 
ITQ  is  to  measure  a  person's  tendency  to  become  immersed. 
It  follows  that  those  who  score  high  on  the  ITQ  should 
correspondingly  score  high  on  the  PQ  (see  figure  14)  .  The 
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no  sound  condition  was  the  only  one  in  which  some 
participants'  PQ  scores  were  lower  than  their  ITQ  scores. 
This  result  was  expected  since  three  of  the  PQ  questions 
deal  specifically  with  audio,  where  the  no  sound  condition 
scores  would  obviously  be  lower. 


Speakers  ITQ  Total  vs  PQ  Total 
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Figure  14.  Plot  of  ITQ  and  PQ  Scores  by  Condition. 

Further  analysis  of  the  questionnaire  data  was  used  to 
determine  what  affect  the  sound  delivery  method  had  on  a 
user's  sense  of  presence  while  in  the  virtual  environment. 
Figure  15  shows  the  average  ITQ  and  PQ  scores  by  sound 
delivery  method.  As  can  be  seen  from  figure  15,  a 
difference  exists  between  the  no  sound  condition  and  any  of 
the  sound  conditions.  Of  interest  here  is  the  similarity 
of  the  speaker  and  headphone  condition.  This  implies  that 
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there  is 

marginal  difference. 

if 

any,  between 

the 

5.1 

surround 

sound 

speakers  and 

the 

headphones . 

Also 

of 

interest 

is  the 

drop  in  the 

PQ 

average  score 

for 

the 

headphones 

with 

subwoofer  condition.  This 

may 

be 

attributed  to  the  lower  ITQ  average,  which  implies  that  the 
participants  in  the  headphones  with  subwoofer  condition 
were  less  likely  to  become  immersed  in  the  environment. 
Analysis  of  variance  testing  of  the  total  PQ  score  against 
the  type  of  sound  delivery  method  resulted  in  F(3,76)  = 
5.918,  p  =  0.001.  Therefore,  sound  plays  an  important  role 
in  aiding  the  user  in  developing  a  sense  of  presence  in  a 
virtual  environment. 
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Headphones  w/ 
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Figure  15.  Average  ITQ  and  PQ  Scores  by  Condition. 


Since  a  significant  difference  was  identified  between 
sound  treatments,  additional  analysis  was  required  to 
determine  where  that  difference  occurred.  To  identify  the 
reason  for  the  difference,  paired-wise  t-tests  between 
treatments  were  utilized  and  results  are  shown  in  table  13. 


This  shows  that  while  sound  makes  a  difference  on  a  user's 
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sense  of  presence,  the  method  of  delivery  does  not. 
Removing  the  sound  questions  results  in  only  a  significant 
difference  in  the  no  sound  versus  headphones  comparison. 


Table  13.  Comparison  of  PQ  Scores  between  Conditions. 


Condition 

df 

t  Critical 

E 

£  (w/o  sound 
questions) 

No  sound  vs  Speakers 

19 

1 . 729 

0.005 

0.170 

No  sound  vs  Headphones 

19 

1 . 729 

0.0006 

0.072 

No  sound  vs  Headphones  w/subwoofer 

19 

1 . 729 

0.015 

0.323 

Speakers  vs  Headphones 

19 

1 . 729 

0.706 

0.771 

Speakers  vs  Headphones  w/subwoofer 

19 

1.729 

0.655 

0.676 

Headphones  vs  Headphones  w/subwoofer 

19 

1 . 729 

0.474 

0.561 

To  further  illustrate  the  effect  of  sound  on  presence 
according  to  the  questionnaires,  figure  16  shows  the 
relationship  of  average  ITQ  and  PQ  scores  to  the  score  of 
the  PQ  without  the  sound  related  questions.  Analysis  of 
the  PQ  and  PQ  without  sound  questions  scores  results  in  no 
significant  difference.  One  might  ask  why  the  no  sound 
condition  score  dropped  when  the  sound  questions  were 
removed  from  the  total.  As  stated  earlier,  the  questions 
are  rated  on  a  one  to  seven  scale.  This  means  that  the 
lowest  possible  answer  still  contributes  a  value  of  one  to 
the  total  score.  Zeroing  the  scoring  on  the  questionnaire 
only  scales  down  the  graphs,  but  the  overall  trends  remain 
the  same.  Also,  some  of  the  participants  in  the  no  sound 
condition  rated  the  audio  questions  higher  than  the  lowest 
possible  score.  Initially,  these  answers  were  thought  to 
be  erroneous.  Further  investigation  revealed  that  some  of 
the  no  sound  condition  participants  were  unaware  they  had 
the  no  sound  condition.  This  is  just  one  example  of  the 
inherent  problems  with  subjective  questionnaires. 
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Figure  16.  Average  Scores  of  ITQ,  PQ,  and  PQ  w/o  Sound 

Questions . 


The  next  step  in  the  questionnaire  analysis  was  the 
comparison  of  the  PQ  scores  to  the  Slater  question  scores. 
A  visual  examination  of  the  graph  of  average  answer  values 
appears  to  show  very  little  difference  (see  figure  17),  but 
a  statistical  analysis  shows  a  significant  difference 
(F(l,75)  =  79.118,  p  =  0.000).  In  spite  of  this 
difference,  the  two  questionnaires  are  highly  correlated  (rg 
=  0.680,  £  =  0.000).  Further  analysis  shows  no 
significance  when  comparing  the  Slater  average  answer  value 
between  conditions  (F(3,76),  p  =  0.151). 

This  indicates  that  the  PQ  may  have  been  the  better 
subjective  measure  of  presence  for  this  particular 
environment.  The  PQ  consisted  of  twenty-four  questions 
ranging  from  the  participant's  sense  of  presence  within  the 
environment  to  immersive  aspects  of  the  environment  (video 
quality,  interface  design  issues,  audio  aspects,  etc.). 
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Slater's  six  questions  deal  entirely  with  the  participant's 
cognitive  recollections  of  the  virtual  environment. 


PQ  Answer  Avg 
®  Slater  Answer  Avg 


No  Sound  Speakers  Headphones  Headphones  w/  Bass 

Condition 


Figure  17.  Average  PQ  and  SQ  Answer  Values. 

Comparing  Physiological  Responses  to  Questionnaires: 

The  first  step  in  this  analysis  consisted  of  performing 
ANOVAs  to  determine  if  any  significant  difference  existed 
between  the  physiological  responses  and  the  PQ  scores.  The 
results  of  the  comparison  are  shown  in  table  14.  The 
results  show  a  significant  difference  between  the  PQ  score 
and  changes  in  physiological  responses  in  the  conditions  of 
no  sound  and  headphones  only,  with  or  without  outliers. 
Elect rodermal  activity  (EDA)  and  heart  rate  (HR)  were  the 
only  responses  to  show  this  effect. 
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Table  14.  PQ  Score  vs  Physiological  Response  by  Condition 


Condition  Change  in  EDA 

df 

F  statistic 

E 

No  sound 

(1,  16) 

5.283 

0.035 

No  sound  w/o  outlier (s) 

(1,  15) 

7.291 

0.016 

Speakers 

(1,  16) 

0.400 

0.536 

Headphones 

\ — 1 

< — 1 

7.035 

0.017 

Headphones  w/o  outlier (s) 

(1,  15) 

0.499 

0.491 

Headphones  w/  Subwoofer 

(1,  16) 

0.222 

0 . 644 

Condition  Change  in  HR 

df 

F  statistic 

E 

No  sound 

(1,  16) 

0.598 

0.451 

No  sound  w/o  outlier (s) 

(1,  15) 

9.259 

0.008 

Speakers 

\ — 1 

< — 1 

1.965 

0.180 

Headphones 

(1,  16) 

4.019 

0.062 

Headphones  w/o  outlier (s) 

(1,  15) 

5 . 142 

0.039 

Headphones  w/  Subwoofer 

(1,  16) 

0.299 

0.592 

Condition  Change  in  TEMP 

df 

F  statistic 

E 

No  sound 

(1,  16) 

1 . 125 

0.305 

No  sound  w/o  outlier (s) 

(1,  15) 

1.574 

0.229 

Speakers 

\ — 1 

< — 1 

0.154 

0.700 

Headphones 

(1,  16) 

0.502 

0.489 

Headphones  w/o  outlier (s) 

(1,  15) 

0.223 

0 . 642 

Headphones  w/  Subwoofer 

(1,  16) 

0.009 

0.925 

Additional  tests  were  conducted  to  identify  other 
factors  correlated  to  the  PQ  scores.  Factors  evaluated 
were  computer  gaming  experience,  caffeine,  age,  gender,  and 
sleep.  Gaming  and  caffeine  were  the  only  two  factors  to 
show  significant  correlation  (rg  =  0.528,  p  =  0.0  and  rg  = 
0.187,  p  =  0.096,  respectively) .  Based  on  these  results, 
a  multi-comparison  ANOVA  comparing  PQ  scores  with  condition 
and  all  two  and  three-way  interactions  of  changes  in  EDA, 
HR,  and  temperature,  gaming,  and  caffeine  was  conducted. 
The  results  are  shown  in  table  15  and  figure  18.  Again,  it 
showed  any  sound  condition  had  an  effect  on  the 
participants'  PQ  score  but  delivery  method  showed  no 
significance . 
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Table  15.  Multi-comparison  ANOVA  Results. 


Conditions 

Estimate 

Std.  Error 

Lower  Bound 

Upper  Bound 

No  sound  -  Speakers 

-20.60 

6.86 

-36.6 

-4.58 

No  sound  -  Headphones 

-22.30 

5.93 

-36.2 

-8.51 

No  sound  -  Headphones  w/  Subwoofer 

-28.90 

6.11 

-43.1 

-14.60 

Speakers  -  Headphones 

-1.75 

6.82 

-17.7 

14.20 

Speakers  -  Headphones  w/  Subwoofer 

-8.28 

7 . 04 

-24.7 

8.15 

Headphones  -  Headphones  w/  Subwoofer 

-6.52 

6.31 

-21.3 

8.21 

No  Sound-Speakers 
No  Sound-Headphones 

No  Sound-Headphones  w/Bass  ( - 

Speakers-Headphones 
Speakers-Headphones  w/Bass 
Headphones-Headphones  w/Bass 

- r“ 

-45  -40 


( - 

(—■ 


T“ 

-35 


— ; - 1 - ; - 1 - 1 - 1 - 1 - ; - r 

-30  -25  -20  -15  -10  -5  0  5  10 

simultaneous  90  %  confidence  limits,  simulation-based  method 
response  variable:  PQ. Total 


) 
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Figure  18.  Graph  of  Multi-comparison  ANOVA  Results. 

A  similar  analysis  of  the  relationship  between 
physiological  responses  and  the  Slater  presence  questions 
revealed  no  significant  difference  on  any  response  or 
condition.  Again,  this  could  be  attributable  to  the 
cognitive  basis  of  the  questions. 

The  last  step  in  the  analysis  was  to  determine  if  any 
correlation  existed  between  the  measured  physiological 
responses  and  the  presence  questionnaire  scores.  Again, 
the  Spearman  Rank  Correlation  was  the  analytical  tool  of 
choice.  A  correlation  exists  between  EDA  and  PQ  score  (rg  = 
0.349,  p  =  0.002)  and  HR  and  PQ  score  (rg  =  0.247,  p  = 
0.028),  but  not  temperature  and  PQ  score.  Similar 
correlation  tests  on  physiological  responses  and  the  Slater 
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presence  questions  (SQ)  score  resulted  in  a  correlation 
between  EDA  and  SQ  score  (rs  =  0.230,  p  =  0.041)  .  This 
again  indicates  that  the  PQ  may  have  been  the  better 
subjective  measure  of  presence  for  this  particular 
environment . 

C.  SUMMARY 

The  analysis  of  the  effect  of  sound  delivery  method 
indicates  that  sound  had  a  significant  effect  on  the  user' s 
sense  of  presence  in  this  virtual  environment.  Visual 
stimuli  alone  had  no  significant  effect  on  EDA  (no  sound 
condition) ,  but  the  addition  of  sound  (speakers, 

headphones,  and  headphones  with  subwoofer)  in  the  virtual 
environment  created  a  significant  effect  on  this  response. 
Electrodermal  activity  is  an  indication  of  arousal,  which 
is  considered  a  primary  dimension  of  emotion  [DETE  98]  . 
The  speaker  delivery  method  was  shown  to  have  a  significant 
effect  on  temperature.  Temperature  has  been  shown  to  fall 
during  emotional  reactions  [GROS  67]  and  is  considered  a 
valid  measure  of  fear  response  [MEEH  00] . 

The  use  of  physiological  responses  as  an  objective 
measure  of  presence  appears  to  be  feasible.  While 
temperature  was  the  only  response  to  be  affected  by  the 
sound  delivery  method,  electrodermal  activity  and  heart 
rate  indicate  a  significant  correlation  to  the  subjective 
PQ  scores.  Electrodermal  activity  was  the  only 

physiological  response  to  correlate  with  the  Slater 
questions  score. 

Presence  questionnaire  results  show  a  significant 
difference  between  no  sound  and  any  sound  condition.  This 
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indicates  that  headphones  elicit  an  equivalent  subjective 
level  of  presence  as  the  high-end  surround  sound  speakers. 
The  addition  of  a  subwoofer  to  the  headphones  had  no 
significant  effect. 

Although  both  the  Witmer  and  Singer  Presence 
Questionnaire  and  Slater' s  six  presence  questions  have  been 
used  as  subjective  means  for  gauging  a  user's  sense  of 
presence,  these  two  questionnaires  do  not  measure  the  same 
aspects  of  presence.  The  PQ  encompasses  more  attributes  of 
the  virtual  environment,  including  sound  (the  independent 
variable  of  this  experiment),  while  Slater's  questions 
concentrate  primarily  on  the  cognitive  aspects  of  the 
environment.  Due  to  these  facts,  the  PQ  is  a  more 
appropriate  subjective  measure  for  this  virtual 
environment . 
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V.  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  SUMMARY 

The  purpose  of  this  study  was  two-fold.  First,  the 
effect  of  sound  delivery  method  on  a  user' s  sense  of 
presence  in  a  virtual  environment  was  determined  based  on 
the  participants'  physiological  responses.  Second,  the 
physiological  responses  were  compared  to  presence 
questionnaire  scores  to  determine  if  a  correlation  exists. 

The  analysis  of  the  effect  of  sound  delivery  method 
indicates  that  sound  had  a  significant  effect  on  the  user' s 
sense  of  presence  in  this  virtual  environment.  Visual 
stimuli  alone  had  no  significant  effect  on  EDA  (no  sound 

condition),  but  the  addition  of  sound  (speakers, 

headphones,  and  headphones  with  subwoofer)  in  the  virtual 
environment  created  a  significant  effect  on  this  response. 
Electrodermal  activity,  especially  the  frequency  and 
amplitude,  is  a  known  indication  of  arousal  [SIMO  99]  [AX 
64]  .  Arousal  is  considered  to  be  one  of  the  two  most 

commonly  cited  dimensions  of  emotion  [DETE  98]  .  Thus,  the 
addition  of  sound  to  a  virtual  environment  affects  the 
level  of  presence  by  inducing  arousal  in  the  user. 

Temperature  was  shown  to  decrease  in  the  speaker 
condition  more  than  in  either  of  the  headphone  conditions. 
Temperature  has  been  shown  to  fall  during  emotional 

reactions  [GROS  67] .  It  is  also  considered  a  valid  measure 
of  fear  response  [MEEH  00]  .  Headphones  with  subwoofer, 
although  not  reaching  significance,  had  the  second  largest 
drop  in  temperature.  This  is  a  potential  indication  that 
the  subwoofer  played  a  part  in  eliciting  fear  in  the  user. 
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This  occurs  as  the  body  withdraws  blood  from  the 
extremities  and  sends  more  to  the  vital  organs  as  needed 
for  "fight  or  flight".  The  advantage  the  surround  sound 
speakers  had  over  any  headphone  condition  was 
spatialization .  This  added  ability  to  localize  sound  may 
have  induced  a  more  pronounced  fear  response. 

The  physiological  responses  of  electrodermal  activity 
EDA  and  HR  showed  significant  correlation  to  the  presence 
questionnaire  scores,  but  temperature  did  not.  This 

follows  the  results  found  in  a  previous  study  by  Meehan 
[MEEH  00]  .  The  use  of  the  physiological  responses  of  EDA 
and  HR  as  an  objective  measure  of  presence  appears  valid. 
However,  the  questionnaires  used  do  not  explicitly  measure 
any  kind  of  emotional  response.  Witmer  and  Singer's 
Presence  Questionnaire,  although  they  consider  immersion  a 
psychological  state  that  is  a  prerequisite  for  presence 
[WITH  98],  measures  how  involving  the  VE  was  or  how  natural 
the  interaction  was  but  not  how  it  made  the  user  feel 
emotionally.  Slater's  questions,  on  the  other  hand,  only 
deal  with  the  cognitive  aspects  of  the  environment  such  as 
how  the  user  remembered  the  experience.  In  order  to 
concretely  link  physiological  responses  to  presence,  a 
questionnaire  that  asks  how  the  environment  made  the  user 
"feel"  (i.e.  afraid,  excited,  anxious,  etc.)  may  be 
required . 

The  results  of  the  subjective  questionnaires  indicate 
that  sound  plays  an  important  part  in  eliciting  presence  in 
the  user.  However,  the  method  of  delivery  of  the  sound  has 
no  significant  effect.  These  findings  indicate  that  the 
use  of  headphones  have  the  same  effect  on  presence  as  a  5.1 
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surround  sound  speaker  system.  The  use  of  headphones  in  a 
virtual  environment  decreases  the  system  cost  and  required 
area  for  setting  up  the  environment,  two  extremely 
important  considerations  for  the  military. 

However,  subjective  questionnaires  have  inherent 
problems.  In  this  study,  some  of  the  no  sound  condition 
participants  rated  the  sound  related  questions  above  the 
minimum  values.  Upon  detection  of  the  higher  values, 
investigation  revealed  that  some  of  the  no  sound 
participants  did  not  realize  they  had  no  sound  during  game 
play.  Such  obviously  erroneous  readings  can  skew  the  data. 
Fortunately  in  this  study,  removing  these  participants' 
results  from  the  database  did  not  dramatically  change  the 
findings . 

The  results  of  this  study  demonstrated  the  inherent 
problems  with  questionnaires.  Automatic  reactions  such  as 
physiological  responses  bypass  these  problems  and  provide  a 
more  reliable  measure  of  presence.  Due  to  these  facts,  it 
is  our  conclusion  that  physiological  responses  are  a  more 
effective  measure  of  presence.  The  lack  of  complete 
correlation  of  physiological  responses  to  subjective 
questionnaires  only  indicated  that  the  questionnaires  did 
not  contain  the  appropriate  questions  to  fully  measure  the 
sense  of  presence. 

B .  RECOMMENDATIONS 

The  scenario  selected  in  Medal  of  Honor:  Allied 
Assault™  was  not  conducive  to  localizing  sound.  It 
contained  many  explosions  that  surrounded  the  player. 
Therefore,  the  necessity  to  localize  sounds  was  not  a 
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requirement  for  this  particular  game  scenario.  A  more 
fitting  scenario  to  determine  the  differences  between 
headphones  and  surround  sound  speakers  would  be  one  that 
required  the  user  to  determine  the  direction  of  sound (s)  in 
order  to  successfully  complete  the  mission. 

The  baseline  physiological  readings  were  taken  upon 
arrival  in  the  lab  after  the  participant  completed  the 
training  scenario  and  the  pre-questionnaire.  This  may  have 
not  allowed  enough  time  for  the  participant's  signals  to 
return  to  a  rest  state  after  walking  to  the  lab  (some  had 
to  walk  further  than  others)  .  A  fifteen-minute  rest  time 
before  taking  baseline  readings  (after  completing  both  the 
training  scenario  and  pre-questionnaire)  might  have  lead  to 
more  accurate  representations  of  baseline  readings.  Also, 
the  participants  read  the  Intelligence  Brief  while  their 
baseline  readings  were  being  recorded.  A  better  way  may 
have  been  to  "isolate"  them  while  taking  baseline  readings. 
Having  the  participant  sit  in  a  relatively  dark  and  silent 
room  may  have  been  more  appropriate. 

During  game  play,  most  of  the  participants  would  ask 
the  observer  questions  about  the  scenario.  This  inevitably 
detracted  from  their  sense  of  presence  in  the  environment. 
Notifying  the  participants  that  no  questions  would  be 
answered  after  game  play  commenced  would  have  minimized 
this  possible  distraction. 

The  experiment  conducted  was  a  between-sub ject s  design 
in  order  to  minimize  any  adverse  effect  due  to  previous 
exposure  to  the  environment.  One  of  the  deciding  factors 
in  using  Medal  of  Honor:  Allied  Assault™  as  the  environment 
was  its  uniqueness.  Having  recently  been  released,  the 
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threat  of  previous  exposure  was  expected  to  be  minimal. 
However,  there  were  four  participants  who  had  played  the 
game  before  the  experiment.  Removing  their  data  from  the 
analysis  resulted  in  no  significant  change  in  the  outcomes. 
This  indicates  that  previous  exposure,  although  shown  to 
have  an  effect  on  presence  in  previous  research  [MEEH  00], 
had  minimal  effect  in  this  experiment.  A  within-sub ject s 
design  may  result  in  more  pronounced  effects  due  to  less 
variability  caused  by  individual  differences. 

All  participants  were  subjected  to  surround  sound 
speaker  sound  while  conducting  the  training  scenario, 
although  at  a  much  lower  level  than  game  play.  While  this 
was  probably  not  a  problem  for  those  that  received  a  sound 
condition,  the  no  sound  participants  may  have  been 
inadvertently  biased  due  to  a  preconceived  expectation  of 
sound.  This  bias  may  have  influenced  their  responses  on 
the  subjective  questionnaires.  Having  all  participants 
complete  the  training  scenario  with  only  verbal  cues  from 
the  experimenters  may  have  been  a  better  method,  thereby 
eliminating  any  unwanted  influence. 

C .  FUTURE  WORK 

Although  this  experiment  shows  that  the  physiological 
responses  of  EDA  and  HR  correlate  with  an  individual's 
subjective  sense  of  presence  while  in  a  virtual 
environment,  further  work  is  required  to  corroborate  these 
findings.  Some  future  work  that  may  be  done  to  corroborate 
this  data  follows. 

One  of  the  drawbacks  of  using  human  participants  is 
the  variability  between  individuals.  In  order  to  minimize 
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this  variability,  large  sample  sizes  are  needed.  Rerunning 
the  experiment,  adding  the  data  to  the  current  data,  and 
then  performing  analysis  on  the  whole  data  set  may  provide 
stronger  evidence  either  for  or  against  physiological 
responses  being  used  as  objective  measures  of  presence. 

Although  the  scenario  chosen  for  this  experiment 
provided  captivating  sound,  it  did  not  provide  many 
localizable  sounds.  The  majority  of  the  sound  came  from 
artillery  explosions  occurring  all  around  the  player. 
Selecting  a  scenario  that  provides  sounds  that  are  more 
localizable  may  show  a  significant  difference  between  sound 
delivery  methods.  The  use  of  a  scenario  with  localizable 
sounds  would  also  permit  the  use  of  head  related  transfer 
functions  (HRTF) .  The  experiment  could  incorporate  HRTFs 
to  also  determine  the  affect  of  sound  delivery  methods  on 
the  user's  sense  of  presence. 

Another  objective  method  of  presence  measurement  is 
postural  response.  Video  of  the  participants  during  game 
play  was  taken  but  not  analyzed  due  to  time  constraints. 
Analysis  of  the  video  could  be  done  to  compare  postural 
responses  to  both  physiological  responses  and  presence 
questionnaire  totals.  Effort  would  be  required  to  develop 
a  metric  for  determining  significant  movement  since  head 
tracking  was  not  employed.  Further  experimentation  using  a 
method  of  tracking  postural  responses  could  also  be  used  to 
corroborate  these  findings. 

This  experiment  concentrated  on  the  effect  of  sound 
delivery  method  on  the  user's  sense  of  presence.  The 
questionnaires  used  concentrated  primarily  on  other  than 
the  sound  aspects  of  the  environment.  Only  three  of  the 


twenty-four  questions  dealt  directly  with  sound.  Altering 
the  immersive  qualities  of  the  environment  by  other  than 
sound  (i.e.  HMD,  big  screen  TV,  CAVE,  etc.)  could  greatly 
affect  the  individual's  sense  of  presence  and  their 
physiological  responses.  Although  this  will  not  aid  in 
determining  which  sound  delivery  method  has  the  greatest 
effect  on  presence,  it  may  aid  in  determining  the 
correlation  between  physiological  responses  and  an 
individual's  sense  of  presence. 

Due  to  the  subjective  nature  of  presence, 
questionnaires  are  the  primary  means  of  measurement.  The 
questionnaires  used  in  this  experiment  are  only  two  of  the 
handful  currently  available.  Running  the  experiment  with  a 
different  presence  questionnaire  may  further  validate  the 
use  of  physiological  responses  as  an  objective  measure  of 
presence.  A  comparison  of  the  different  questionnaires  may 
also  contribute  to  this  body  of  work.  Furthermore,  since 
physiological  responses  are  correlates  of  emotion,  a 
questionnaire  asking  emotion  related  questions  might  be 
more  appropriate  to  determine  the  link  between 
physiological  responses  and  presence.  Development  of  such 
a  questionnaire  would  most  assuredly  be  helpful  in  this 
endeavor . 
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APPENDIX  A.  RAW  DATA 


Condition 

No  Sound 
No  Sound 
No  Sound 
No  Sound 
No  Sound 
No  Sound 
No  Sound 
No  Sound 
No  Sound 
No  Sound 
No  Sound 
No  Sound 
No  Sound 
No  Sound 
No  Sound 
No  Sound 
No  Sound 
No  Sound 
No  Sound 
No  Sound 


Speakers 
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Speakers 
Speakers 
Speakers 
Speakers 
Speakers 
Speakers 
Speakers 
Speakers 
Speakers 
Speakers 
Headphones 
Headphones 
Headphones 
Headphones 
Headphones 
Headphones 
Headphones 
Headphones 
Headphones 
Headphones 
Headphones 
Headphones 
Headphones 
Headphones 
Headphones 
Headphones 
Headphones 
Headphones 
Headphones 
Headphones 
HP  w/Subwoofer 
HP  w/Subwoofer 
HP  w/Subwoofer 
HP  w/Subwoofer 
HP  w/Subwoofer 
HP  w/Subwoofer 
HP  w/Subwoofer 
HP  w/Subwoofer 
HP  w/Subwoofer 
HP  w/Subwoofer 
HP  w/Subwoofer 
HP  w/Subwoofer 
HP  w/Subwoofer 
HP  w/Subwoofer 
HP  w/Subwoofer 
HP  w/Subwoofer 
HP  w/Subwoofer 
HP  w/Subwoofer 
HP  w/Subwoofer 
HP  w/Subwoofer 


BLTEMP 

BLHR 

CEDA 

CTEMP 

CHR 

DEDA 

DTEMP 

DHR 

QEDA 

QTEMP 

QHR 

Sieep 

Caffein 

Gaming 

Gender 
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APPENDIX  B.  EXPERIMENT  PROTOCOL 


I .  Consent  forms 

A.  Have  participant  read  and  sign  consent  forms. 

B.  Assign  participant  a  subject  ID  and  Condition. 

C.  Condition  values: 

1  .  1  -  No  sound 

2.2-  Speakers 

3 .  3  -  Headphones  without  subwoofer 

4 .  4  -  Headphones  with  subwoofer 

II.  Initialize  ITQ  on  computer  1 

A.  Logon  to  computer. 

B.  Start  MediaLab  by  double-clicking  on  the  desktop 
icon . 

C.  Click  on  Run  in  the  menu  bar. 

1.  click  on  Select  and  run  an  experiment. 

2.  select  c:\Documents  and 

Sett ings\ Administrator \ Desktop \QUE ST lONNAIRES 

3.  select  AudioEf f ectOnPresence 

4 .  click  on  Open 

5.  Go  to  step  III. 

III.  Immersive  Tendencies  Questionnaire 

A.  Input  subject  ID  and  condition  as  assigned  in  step 
I  above . 

B.  Have  participant  complete  ITQ  on  computer  I. 

1.  Have  participant  answer  questions. 

2.  If  participant  should  come  to  you  about  any 
health  concerns,  determine  if  concern 
precludes  participant  from  participating  in 
the  experiment.  If  NO,  have  participant 
continue  with  questionnaire  by  pressing  1.  If 
YES,  press  2  on  computer  and  thank  the 
participant  for  volunteering. 

C.  When  participant  finishes  the  ITQ,  move  him  to 
computer  2  for  practice  (step  IV) . 

IV.  Medal  of  Honor  Practice 

A.  Logon  to  computer  in  the  Multimedia  Lab. 

1.  Start  MQHAA  by  doubleclicking  on  the  MOHAA 
shortcut  on  the  desktop  (ensure  MOHAA  disc  1 
is  in  the  drive) 

2.  Press  any  button  to  exit  the  introduction 
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B.  Click  on  the  wall  map 

1.  Click  on  the  Basic  Training  clipboard. 

2.  Have  the  participant  follow  the  DI's 
instructions . 

C.  Once  the  participant  is  finished  with  the  tutorial, 
go  to  step  VI . 


V.  Biograph  setup 

A.  Logon  to  computer  in  side  room. 

B.  Double  click  on  the  BioGraph  2.1  icon. 

C.  Click  on  OK 

D.  Click  on  Load  a  Display  Screen. 

E.  Under  Categories,  select  Pilot  Study  (if  not 
selected)  and  then  select  Pilot  Study  under  Display 
Screens.  Click  on  Load. 

F.  Click  on  Start  New. 

1.  Highlight  Thesis,  Audio 

2.  Ensure  battery  is  at  least  30%.  If  not, 
replace  batteries 

G.  Click  on  Start. 

H.  Once  the  session  is  over,  click  Stop  (button  with  a 
black  square  on  it) . 

I.  Click  YES  on  the  Do  you  want  to  save  the  recording 
for  client  Thesis,  Audio? 

1.  Enter  sub#cond*  for  the  description  (where  # 
is  the  participant's  ID  number  and  *  is  the 
condition  number) . 

2.  Press  OK. 

3.  Copy  the  session  to  our  PhysData  file 

a.  Click  on  File  in  the  menu  bar 

b.  Select  Load  Session  from  the  drop  down 
menu . 

c.  Highlight  the  session  you  want  to  copy. 

d.  Click  on  the  Copy  Session  to:  button. 

e.  Type  in  C:\PhysData  in  the  Enter 
destination  for  client  data:  dialog  box 

f.  Click  OK 

J.  Ensure  sensor  cables  on  in  the  right  port  on  the 
ProCompT . 

1  .  Port  A:  EKG 

2.  Port  E:  GSR 

3.  Port  F:  Temperature 

4.  Port  G:  BVP 
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VI .  Attach  sensors  to  participant 

A.  All  sensors  (except  EKG)  go  on  the  participants 
left  hand. 

1 .  Temperature  sensor  attaches  to  the 
participant's  left  pinky  finger  using  the 
Velcro.  The  sensor  should  be  on  the  pad  of 
the  finger. 

2.  GSR  sensors  go  on  the  participant's  middle 
finger . 

a.  One  sensor  is  wrapped  around  the  base  of 
the  finger. 

b.  One  sensor  is  wrapped  around  the  next 
joint  up  from  the  base. 

3.  BVP  sensor  goes  on  the  participant's  thumb 
with  the  decal  pointing  away  from  the  thumb. 
Use  the  two  black  elastic  loops  to  secure  the 
sensor  on  the  thumb. 

B.  EKG  cables  attach  to  both  forearms. 

1.  The  blue  connector  goes  on  a  pad  attached  to 
the  inside  of  the  participant's  left  wrist. 

2.  The  black  connector  goes  on  a  pad  attached  to 
the  participant's  forearm  just  below  the  left 
elbow  joint. 

3.  The  yellow  connector  goes  on  a  pad  attached  to 
the  inside  of  the  participant's  right  wrist. 

VII.  MOHAA  Experiment  setup 

A.  Logon  to  computer. 

B.  Set  the  audio  volume  for  the  condition  to  be  tested 

1.  Go  to  Start /Settings/Control  Panel  and  click 
on  the  AudioHQ  icon. 

2.  Click  on  the  Mixer  icon. 

a.  Eor  the  No  Sound  condition  (condition  1) 
check  the  mute  box  under  the  volume 
slider . 

b.  Eor  the  Speakers  condition  (condition  2), 
ensure  the  following  settings  on  the 
mixer  panel : 

1 .  Volume :  45% 

2 .  Bass :  75% 

3.  All  other  sliders:  50%  (the 
last  two  sliders  in  the  panel 
should  be  muted) . 

4.  Click  on  the  Advanced  mode 
button  at  the  bottom  left  of 
the  mixer  window  (if  it  is 
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already  in  advanced  mode,  the 
button  will  be  labeled  Basic 
mode) . 

a.  Set  speakers  to  5.1  speakers 

c.  For  the  Headphone  without  subwoofer 
condition  (condition  3) ,  ensure  the 
following  settings  on  the  mixer  panel: 

1.  Volume:  100% 

2 .  Bass :  22% 

3.  All  other  sliders:  50%  (the 
last  two  sliders  in  the  panel 
should  be  muted) . 

4.  Click  on  the  Advanced  mode 
button  at  the  bottom  left  of  the 
mixer  window  (if  it  is  already 
in  advanced  mode,  the  button 
will  be  labeled  Basic  mode) . 

a.  Set  speakers  to  Headphones 

b.  Turn  off  all  the  Genelec 
speakers  including  the 
subwoofer . 

d.  For  the  Headphones  with  subwoofer 
condition  (condition  4),  ensure  the 
following  settings  on  the  mixer  panel: 

1.  Volume:  100% 

2 .  Bass :  22% 

3.  All  other  sliders:  50%  (the 
last  two  sliders  in  the  panel 
should  be  muted) . 

4.  Click  on  the  Advanced  mode 
button  at  the  bottom  left  of 
the  mixer  window  (if  it  is 
already  in  advanced  mode,  the 
button  will  be  labeled  Basic 
mode) . 

a.  Set  speakers  to  Headphones 

b.  Turn  off  all  the  Genelec 
speakers  except  the 
subwoofer . 

C.  Click  on  the  in/out  box  on  the  desk 

1.  Click  on  the  Load/Save  trays. 

2.  Highlight  "Omaha  Beach  -  The  Landing"  with  the 
elapsed  time  of  0:05:52. 

3.  Click  on  Load. 

D.  Inform  the  participant  not  to  press  "Continue" 
until  told  to  do  so. 


94 


E  . 


F  . 


Start  the  video  camera 

1.  Turn  power  button  to  Cam  Corder 

2.  Ensure  tape  is  loaded 

3.  Ensure  view  has  both  computer  monitor  and 
participant 

4.  Start  the  camera  just  prior  to  the  participant 
starting  the  game. 

When  the  participant  is  finished,  direct  him/her 
back  to  the  first  computer  to  answer  the  PQ. 
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APPENDIX  C.  INBRIEF  SCRIPT 


GENERAL : 

The  following  Intel  brief  was  provided  to  each 
participant  prior  to  participation  in  this  experiment.  The 
Intel  brief  appears  in  the  same  format  used  for  the 
experiment  and  does  not  follow  the  standard  thesis  format 
utilized  in  this  thesis. 
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INTEL  BRIEF 
Normandy  Invasion 
Omaha  Beach ^  Charlie  Sector 
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SASOJOW  WRlAi.  PMOuAw*  ?J  MJI»  -.'.i 


You  are  2’^'^  Lt .  Mike  Powell,  2"^^  Ranger  Battalion,  an  integral 
part  of  the  Omaha  Beach  invasion  of  June  1944.  You  have  made  it 
past  the  open  beach  alive  and  are  at  the  "shingle".  You  are 
required  to  retrieve  bangalores  and  bring  them  back  to  the 
shingle  to  clear  the  wire.  The  bangalores  are  off  to  your  left 
and  a  little  towards  the  water.  You  will  have  to  expose 
yourself  to  enemy  fire  in  order  to  retrieve  the  bangalores,  but 
your  battalion  can't  proceed  without  them. 

Once  the  wire  is  cleared,  make  your  way  to  the  base  of  the 
bunker.  Once  at  the  bunker,  Capt .  Schmuck  will  send  two  brave 
souls  out  to  reach  the  trench  at  the  base  of  the  second  bunker. 
If  they  do  not  make  it,  you  will  have  to  do  it.  Between  the 
bunkers  and  the  trench  is  a  minefield  -  choose  your  path 
carefully,  but  quickly.  On  the  hill  between  the  bunkers  are  a 
couple  of  machine  gun  nests  that  are  waiting  for  you  to  get  out 
in  the  open. 

Once  you  make  it  to  the  trench,  you  must  infiltrate  the  bunker, 
take  it  over,  and  take  out  the  machine  gun  nests  and  second 
bunker.  Make  your  way  through  the  bunker  carefully.  You  never 
know  when  you  will  run  into  a  Nazi. 
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APPENDIX  D.  CONSENT  FORMS 


GENERAL : 

The  forms  in  the  appendix  appear  in  the  same  format  used 
for  the  experiment  and  do  not  follow  the  standard  thesis  format 
utilized  in  this  thesis.  This  appendix  consists  of  three 
documents:  Consent  Form,  Minimal  Risk  Consent  Statement,  and 
the  Privacy  Act  Statement.  Each  subject  is  required  to  read  and 
sign  these  documents  prior  to  participating  in  the  experiment. 
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PARTICIPANT  CONSENT  FORM 


1.  Introduction.  You  are  invited  to  participate  in  a  study  exploring  how  sound  affects  the  human 
physiology.  This  research  is  aimed  at  improving  sound  systems  in  virtual  environments.  You  will  be 
playing  a  scenario  in  Medal  of  Honor:  Allied  Assault.  After  the  scenario  you  will  complete  a  presence 
questionnaire  to  indicate  how  present  you  felt  in  the  environment.  Your  recorded  data  will  be  used  in  a 
effort  to  determine  if  a  person’s  sense  of  presence  is  correlated  with  the  body’s  physiological  responses 

2.  Background  Information.  Data  is  being  collected  by  the  Naval  Postgraduate  School’s  Human 
System’s  Integration  Laboratory  for  use  in  developing  virtual  environments. 


3.  Procedures.  If  you  agree  to  participate  in  this  study,  the  researcher  will  explain  the  tasks  in  detail. 
Auditory  stimuli  will  be  presented  over  different  speaker  configurations  while  visual  stimuli  are 
presented  over  the  same  delivery  means.  You  will  be  connected  to  a  computer  via  a  junction  box  and 
several  wires  that  will  be  harmlessly  attached  your  body.  You  will  use  the  mouse  and  keyboard  to  play 
a  Medal  of  Honor  game  scenario.  The  intent  is  for  you  to  play  the  game  to  the  best  of  your  ability.  The 
entire  task  will  take  approximately  30  minutes. 


4.  Risks  and  Benefits.  Because  this  research  involves  minimal  risks  to  individuals  with  cardiac  ris 


ctors,  we  request  that  IF  YOU  CONSIDER  YOURSELF  AS  SUCH,  PLEASE  INFORM 
PERIIMENT  ADMINISTRATOR  AT  ONCE,  and  NOT  PROCEED  ANY  FURTHER.  The 


iac  risj^ 
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benefits  to  the  participants  will  be  to  contribute  to  current  research  in  advancing  presence  in  virtual 
environments  and  in  human-computer  interaction. 


5.  Compensation.  No  tangible  reward  will  be  given.  A  copy  of  the  results  will  be  available  to  you  at  the 
conclusion  of  the  experiment. 


6.  Confidentiality.  The  records  of  this  study  will  be  kept  confidential.  No  information  will  be  publicly 
accessible  which  could  identify  you  as  a  participant. 

7.  Voluntary  Nature  of  the  Study.  If  you  agree  to  participate,  you  are  free  to  withdraw  from  the  study  at 
any  time  without  prejudice.  You  will  be  provided  a  copy  of  this  form  for  your  records. 


8.  Points  of  Contact.  If  you  have  any  further  questions  or  comments  after  the  completion  of  the  study, 
you  may  contact  the  research  supervisor.  Dr.  Russell  Shilling  (831)  656-2543  shilling@cs.nps.navy.mil. 


9.  Statement  of  Consent.  I  have  read  the  above  information.  I  have  asked  all  questions  and  have  had  my 
questions  answered.  I  agree  to  participate  in  this  study. 


Participant’s  Signature 


Date 


Researcher’s  Signature 


Date 
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NAVAL  POSTGRADUATE  SCHOOL,  MONTEREY,  CA  93943 
MINIMAL  RISK  CONSENT  STATEMENT 


Participant: 

VOLUNTARY  CONSENT  TO  BE  A  RESEARCH  PARTICIPANT  IN:  SOUND  SYSTEM  COMPLEXITY  AND 
ITS  EFFECT  ON  THE  USER’S  SENSE  OF  PRESENCE  IN  A  VIRTUAL  ENVIRONMENT. 

1.  I  have  read,  understand  and  been  provided  "Information  for  Participants"  that  provides  the  details  of  the 
below  acknowledgments. 

2.  I  understand  that  this  project  involves  research.  An  explanation  of  the  purposes  of  the  research,  a 
description  of  procedures  to  be  used,  identification  of  experimental  procedures,  and  the  extended  duration  of 
my  participation  have  been  provided  to  me. 

3.  I  understand  that  this  project  does  not  involve  more  than  minimal  risk.  I  have  been  informed  of  any 
reasonably  foreseeable  risks  or  discomforts  to  me. 

4.  I  have  been  informed  of  any  benefits  to  me  or  to  others  that  may  reasonably  be  expected  from  the 
research. 

5.  I  have  signed  a  statement  describing  the  extent  to  which  confidentiality  of  records  identifying  me  will  be 
maintained. 

6.  I  have  been  informed  of  any  compensation  and/or  medical  treatments  available  if  injury  occurs  and  if  so, 
what  they  consist  of,  or  where  further  information  may  be  obtained. 

7.  I  understand  that  my  participation  in  this  project  is  voluntary,  refusal  to  participate  will  involve  no  penalty 
or  loss  of  benefits  to  which  I  am  otherwise  entitled.  I  also  understand  that  I  may  discontinue  participation  at 
any  time  without  penalty  or  loss  of  benefits  to  which  I  am  otherwise  entitled. 

8.  I  understand  that  the  individual  to  contact  should  I  need  answers  to  pertinent  questions  about  the  research 
is  Professor  Russell  Shilling,  Principal  Investigator,  and  about  my  rights  as  a  research  participant  or 
concerning  a  research  related  injury  is  the  Modeling  Virtual  Environments  and  Simulation  Chairman.  A  full 
and  responsive  discussion  of  the  elements  of  this  project  and  my  consent  has  taken  place. 

Medical  Monitor:  Right  Surgeon,  Naval  Postgraduate  School 


Signature  of  Principal  Investigator  Date  Signature  of  Volunteer  Date 


Signature  of  Witness 


Date 
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PRIVACY  ACT  STATMENT 


NAVAL  POSTGRADUATE  SCHOOL,  MONTEREY,  CA  93943 
PRIVACY  ACT  STATEMENT 

1.  Authority:  Naval  Instruction 

2.  Purpose:  DETERMINE  SOUND  SYSTEM  COMPEEXITY  AND  ITS  EEEECT 
ON  THE  USER’S  SENSE  OE  PRESENCE  IN  A  VIRTUAE  ENVIRONMENT 

3.  Use:  Physiological  response  data  will  be  used  for  statistical  analysis  by  the 
Departments  of  the  Navy  and  Defense,  and  other  U.S.  Government  agencies, 
provided  this  use  is  compatible  with  the  purpose  for  which  the  information  was 
collected.  The  Naval  Postgraduate  School  in  accordance  with  the  provisions  of  the 
Ereedom  of  Information  Act  may  grant  use  of  the  information  to  legitimate  non¬ 
government  agencies  or  individuals. 

4.  Disclosure/Confidentiality: 

a.  I  have  been  assured  that  my  privacy  will  be  safeguarded.  I  will  be  assigned  a 
control  or  code  number,  which  thereafter  will  be  the  only  identifying  entry  on 
any  of  the  research  records.  The  Principal  Investigator  will  maintain  the  cross- 
reference  between  name  and  control  number.  It  will  be  decoded  only  when 
beneficial  to  me  or  if  some  circumstances,  which  are  not  apparent  at  this  time, 
would  make  it  clear  that  decoding  would  enhance  the  value  of  the  research  data. 
In  all  cases,  the  provisions  of  the  Privacy  Act  Statement  will  be  honored. 

b.  I  understand  that  a  record  of  the  information  contained  in  this  Consent  Statement 
or  derived  from  the  experiment  described  herein  will  be  retained  permanently  at 
the  Naval  Postgraduate  School  or  by  higher  authority.  I  voluntarily  agree  to  its 
disclosure  to  agencies  or  individuals  indicated  in  paragraph  3  and  I  have  been 
informed  that  failure  to  agree  to  such  disclosure  may  negate  the  purpose  for 
which  the  experiment  was  conducted. 

c.  I  also  understand  that  disclosure  of  the  requested  information  is  voluntary. 


Signature  of  Volunteer  Name,  Grade/Rank  (if  applicable)  DOB  Date 


Signature  of  Witness  Date 
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APPENDIX  E.  QUESTIONNAIRES 


GENERAL 

The  items  in  this  appendix  appear  in  the  same  format 
utilized  for  the  experiment  and  thus  do  not  conform  to  the 
standard  thesis  format  utilized  in  the  chapters  of  this 
document.  This  appendix  consists  of  two  documents:  Immersive 

Tendencies  Questionnaire  (ITQ)  and  a  Presence  Questionnaire 
(PQ)  .  The  PQ  is  a  combination  of  two  previously  used  presence 
questionnaires  (Witmer  &  Singer' s  questionnaire  and  Slater' s 
questionnaire) . 
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Name 

Question  Wording  /  Fiie  Name 

Sleep 

How  much  sleep  did  you  get  last  night? 

<2  hours 

Caffein 

How  many  cups  of  coffee  or  caffeinated  drinks  have  you  had 
today? 

0 

Gaming 

What  level  of  experience  do  you  have  with  first  person  shooter 
games? 

No  experience 

Gender 

What  is  your  gender? 

Male 

Age 

<25 

Hearing 

Do  you  have  any  significant  hearing  loss? 

Yes 

Level 

What  is  your  level  of  hearing  loss  in  dB? 

<5  dB 

Medical 

Do  you  have  any  health  problems  that  may  preclude  you  from 
participating  in  this  experiment? 

Yes 

Concern 

Please  inform  the  experimenter  of  your  health  concerns. 

Continue 

Movies 

Do  you  easily  become  deeply  involved  in  movies  or  TV  dramas? 

NEVER 

TvBook 

Do  you  ever  become  so  involved  in  a  television  program  or  book 
that  people  have  problems  getting  your  attention? 

NEVER 

Alert 

How  mentally  alert  do  you  feel  at  this  time? 

NOT  ALERT 

MovAwar 

Do  you  ever  become  so  involved  in  a  movie  that  you  are  not 
aware  of  things  happening  around  you? 

NEVER 

Charact 

How  frequently  do  you  find  yourself  closely  identifying  with  the 
characters  in  a  story  line? 

NEVER 

VidGame 

Do  you  ever  become  so  involved  in  a  video  game  that  it  is  as  if 
you  are  inside  the  game  rather  than  moving  a  joystick  and 
watching  the  screen? 

NEVER 

FitToda 

How  physically  fit  do  you  feel  today? 

NOT  FIT 

BlockOu 

How  good  are  you  at  blocking  out  external  distractions  when  you 
are  involved  in  something? 

NOT  VERY  GOOD 

WatcGam 

When  watching  sports,  do  you  ever  bee  ome  so  involved  in  the 
game  that  you  react  as  if  you  were  one  of  the  players? 

NEVER 

DayDrea 

Do  you  ever  become  so  involved  in  a  daydream  that  you  are  not 
aware  of  things  happening  around  you? 

NEVER 

Dreams 

Do  you  ever  have  dreams  that  are  so  reai  that  you  feel 
disoriented  when  you  awake? 

NEVER 

Sports 

When  playing  sports,  do  you  ever  become  so  involved  in  the 
game  that  you  lose  track  of  time? 

NEVER 
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Answer  Choices 


2-4  hours 

4-6  hours 

6-8  hours 

>8  hours 

none 

none 

2 

3 

>3 

none 

none 

1 

Novice 

Average 

Experienced 

Expert 

none 

none 

Female 

none 

none 

none 

none 

none 

26-30 

31-35 

36-40 

>40 

none 

none 

No 

none 

none 

none 

none 

none 

5-10  dB 

10-15  dB 

>15dB 

none 

none 

none 

No 

none 

none 

none 

none 

none 

End 

none 

none 

none 

none 

none 

none 

none 

OCCASIONALLY 

none 

none 

OFTEN 

none 

none 

OCCASIONALLY 

none 

none 

OFTEN 

none 

none 

MODERATELY 

none 

none 

FULLY  ALERT 

none 

none 

OCCASIONALLY 

none 

none 

OFTEN 

none 

none 

OCCASIONALLY 

none 

none 

OFTEN 

none 

none 

OCCASIONALLY 

none 

none 

OFTEN 

none 

none 

MODERATELY 

none 

none 

EXTREMELY  FIT 

none 

none 

SOMEWHAT 

none 

none 

VERY  GOOD 

none 

none 

OCCASIONALLY 

none 

none 

OFTEN 

none 

none 

OCCASIONALLY 

none 

none 

OFTEN 

none 

none 

OCCASIONALLY 

none 

none 

OFTEN 

none 

none 

OCCASIONALLY 

none 

none 

OFTEN 

Name 

Question  Wording  /  Fiie  Name 

Answer  Choices 

Concent 

How  well  do  you  concentrate  on  enjoyable  activities? 

NOT  AT  ALL 

none 

none 

MODERATELY 

none 

none 

VERY  WELL 

PlayVG 

How  often  do  you  play  arcade  or  video  games  (Often  should  be 
taken  to  mean  every  day  or  every  two  days,  on  average)? 

NEVER 

none 

none 

OCCASIONALLY 

none 

none 

OFTEN 

TvFight 

Have  you  ever  gotten  excited  during  a  chase  or  fight  scene  on 

TV  or  in  the  movies? 

NEVER 

none 

none 

OCCASIONALLY 

none 

none 

OFTEN 

Scared 

Have  you  ever  gotten  scared  by  something  happening  on  TV  or 
in  the  movies? 

NEVER 

none 

none 

OCCASIONALLY 

none 

none 

OFTEN 

Fearful 

Have  you  ever  remained  apprehensive  or  fearful  long  after 
watching  a  horror  movie? 

NEVER 

none 

none 

OCCASIONALLY 

none 

none 

OFTEN 

LosTack 

Do  you  ever  become  so  involved  in  doing  something  that  you 
lose  all  track  of  time? 

NEVER 

none 

none 

OCCASIONALLY 

none 

none 

OFTEN 

End 

You  are  done  with  the  pre-questionnaire.  Please  inform  the 
experimenter  to  continue  with  the  next  part  of  the  experiment. 

none 

none 

none 

none 

none 

none 

none 

Control 

How  much  were  you  able  to  control  events? 

NOT  AT  ALL 

none 

none 

SOMEWHAT 

none 

none 

COMPLETELY 

Res-Env 

How  responsive  was  the  environment  to  actions  that  you  initiated 
(or  performed)? 

NOT  AT  ALL 

none 

none 

SOMEWHAT 

none 

none 

COMPLETELY 

Natural 

How  natural  did  your  interactions  with  the  environment  seem? 

NOT  AT  ALL 

none 

none 

SOMEWHAT 

none 

none 

COMPLETELY 

VisAsp 

How  much  did  the  visual  aspects  of  the  environment  involve  you? 

NOT  AT  ALL 

none 

none 

SOMEWHAT 

none 

none 

COMPLETELY 

Slates 

Consider  your  memory  of  being  in  the  virtual  environment.  How 
similar  in  terms  of  the  structure  of  the  memory  is  this  to  the 
structure  of  the  memory  of  other  places  you  have  been  today? 

NOT  AT  ALL 

none 

none 

SOMEWFIAT 

none 

none 

VERY  MUCH  SO 

AudAsp 

How  much  did  the  auditory  aspects  of  the  environment  involve 
you? 

NOT  AT  ALL 

none 

none 

SOMEWHAT 

none 

none 

COMPLETELY 

Mechani 

How  natural  was  the  mechanism  that  controlled  movement 
through  the  environment? 

NOT  AT  ALL 

none 

none 

SOMEWHAT 

none 

none 

COMPLETELY 

SensOb] 

How  compelling  was  your  sense  of  objects  moving  through 
space? 

NOT  AT  ALL 

none 

none 

SOMEWHAT 

none 

none 

COMPLETELY 

Slates 

When  you  think  back  on  your  experience,  do  you  think  of  the 
virtual  environment  more  as  images  that  you  saw  or  more  as 
somewhere  you  visited? 

IMAGES  1  SAW 

none 

none 

none 

none 

none 

Slates 


Res -Act 


How  much  did  your  experience  in  the  Virtual  Environment  seem 
consistent  with  your  real  world  experiences? 


NOT  AT  ALL  INCONSISTENT  none 


During  your  experience,  did  you  feel  like  you  were  just  sitting  in 
the  lab  using  a  mouse  to  interact  with  a  computer,  or  did  the 
virtual  battlefield  ovenwhelm  you? 


Were  you  able  to  anticipate  what  would  happen  next  in  response 

to  the  actions  that  you  performed?  NOT  AT  ALL 


MOST  OF  THE  TIME  I 

REALIZED  I  WAS  IN  THE  LAB  none 


SOMEWHAT 

INCONSISTENT  none 


SOMETIMES  THE 
BATTLEFIELD 
OVERWHELMED  ME  none 


SOMEWHAT 


VERY 

none  INCONSISTENT 


none  COMPLETELY 
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Name 

Question  Wording  /  Fiie  Name 

Answer  Choices 

VisSur 

How  completely  were  you  able  to  actively  survey  or  search  the 
environment  using  vision? 

NOT  AT  ALL 

none 

none 

SOMEWHAT 

none 

none 

COMPLETELY 

Slate2 

To  what  extent  wee  there  times  during  the  experience  when  the 
virtual  battlefield  became  reality  for  you  and  you  forgot  about  the 
real  world  of  the  laboratory? 

AT  NO  TIME 

none 

none 

SOMETIMES 

none 

none 

uSBIIIHIH 

Slate4 

During  the  time  of  the  experience,  which  was  strongest  on  the 
whole?  Your  sense  of  being  on  the  virtual  battlefield  or  being  in 
the  real  world  of  the  laboratory. 

THE  REAL  WORLD  (THE 

LAB) 

none 

none 

none 

none 

none 

THE  VIRTUAL 
ENVIRONMENT 

IdeSoun 

How  well  could  you  identify  sounds? 

NOT  AT  ALL 

none 

none 

SOMEWHAT 

none 

none 

COMPLETELY 

LocSoun 

How  well  could  you  localize  sounds? 

NOT  AT  ALL 

none 

none 

SOMEWHAT 

none 

none 

COMPLETELY 

SensMov 

How  compelling  was  your  sense  of  moving  around  inside  the 
virtual  battlefield? 

NOT  COMPELLING 

none 

none 

MODERATELY 

COMPELLING 

none 

none 

VERY 

COMPELLING 

ExmObj 

How  closely  were  you  able  to  examine  objects? 

NOT  AT  ALL 

none 

none 

PRETTY  CLOSE 

none 

none 

VERY  CLOSE 

ExObjVi 

How  well  could  you  examine  objects  from  multiple  viewpoints? 

NOT  AT  ALL 

none 

none 

SOMEWHAT 

none 

none 

EXTENSIVELY 

Slatel 

Rate  your  sense  of  being  in  the  Virtual  Environment  (7 
represents  your  normal  experience  of  being  in  a  place) 

NOT  AT  ALL 

none 

none 

SOMEWHAT 

none 

none 

VERY  MUCH 

Involve 

How  involved  were  you  in  the  virtual  environment  experience? 

NOT  INVOLVED 

none 

none 

MILDLY  INVOLVED 

none 

none 

VERY  INVOLVED 

Delay 

How  much  delay  did  you  experience  between  your  actions  and 
expected  outcomes? 

NO  DELAYS 

none 

none 

MODERATE 

DELAYS 

none 

none 

LONG  DELAYS 

AdjExp 

How  quickly  did  you  adjust  to  the  virtual  environment 
experience? 

NOT  AT  ALL 

none 

none 

SLOWLY 

none 

none 

LESS  THAN  ONE 
MINUTE 

Proficl 

How  proficient  in  moving  and  interacting  with  the  virtual 
environment  did  you  feel  at  the  end  of  the  experience? 

NOT  PROFICIENT 

none 

none 

REASONABLY 

PROFICIENT 

none 

none 

VERY 

PROFICIENT 

DIspQua 

How  much  did  the  visual  display  quality  interfere  or  distract  you 
from  performing  assigned  tasks  or  required  activities? 

NOT  AT  ALL 

none 

none 

SOMEWHAT 

INTERFERRED 

none 

none 

PREVENTED 

PERFORMANCE 

ContDev 

How  much  did  the  control  devices  interfere  with  the  performance 
of  assigned  tasks  or  with  other  activities? 

NOT  AT  ALL 

none 

none 

SOMEWHAT 

INTERFERRED 

none 

none 

GREATLY 

INTERFERRED 

Concen 

How  well  could  you  concentrate  on  the  assigned  tasks  or 
required  activities  rather  than  on  mechanisms  used  to  perform 
those  tasks  or  activities? 

NOT  AT  ALL 

none 

none 

SOMEWHAT 

none 

none 

COMPLETELY 

Comfoc 

Were  there  moments  during  the  virtual  environment  experience 
when  you  felt  completely  focused  on  the  task  or  enviroment? 

NOT  AT  ALL 

none 

none 

OCCASIONALLY 

none 

none 

FREQUENTLY 

Inform 

Was  the  information  provided  through  the  different  senses  in  the 
virtual  environment  (e.g.  vision,  hearing,  touch)  consistent? 

NOT  CONSISTENT 

none 

none 

SOMEWHAT 

CONSISTENT 

none 

none 

VERY 

CONSISTENT 

End2 

This  completes  the  experiment.  Thank  you  for  your  participation. 

none 

none 

none 

none 

none 

none 

none 
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APPENDIX  F.  EQUIPMENT  SPECIFICATIONS 


Dell  Dimension  8100  (2  used) : 


CPU 

Intei  Pentium  4  1.7GHz 

RAM 

256MB 

Hard  drive 

60GB 

Operating  System 

Windows  2000 

Alienware  Majestic  12  (1  used) : 


CPU 

Intei  Pentium  4  1.8GHz 

RAM 

512MB 

Hard  drive 

40GB 

Operating  System 

Windows  XP 

Genelec  1031A  Bi-amplif ied  Speaker  (Five  used) : 


Free  fieid  frequency  response  of  system: 

48  Hz  -  22  kHz  (±  2  dB) 

Harmonic  distortion  at  90  dB  SPL  @  1  m  on  axis: 

Freq:  50... 100  Hz 

<  1% 

>  100  Hz 

<  0.5% 

Drivers: 

Bass 

210  mm  (8")  cone 

Trebie 

25  mm  (1")  metai  dome 

Bass  ampiifier  output  power  with  an  80hm  ioad: 

120  W 

Trebie  ampiifier  output  power  with  an  80hm  ioad: 

Long  term  output  power  is  limited  by  driver  unit  protection  circuitry. 

120  W 

Signai  to  Noise  ratio,  referred  to  fuii  output: 

Bass  >  1 00  dB 
Trebie  >  100  dB 

Genelec  1094A  Active  Subwoofer  System  (1  used) : 


Free  fieid  frequency  response  of  system  (±  2.5 
dB): 

29  -  80  Hz 

Harmonic  distortion  at  1 00  dB  SPL  @  1  m  on  axis 
in  haif  space  (30. ..100  Hz): 

<  3% 

Drivers: 

385  mm  (15") 

Short  term  ampiifier  output  power: 

400  W  (8  Ohm) 

Signai  to  Noise  ratio,  referred  to  fuii  output: 

>  100  dB 
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Sennheiser  Headphones  model  570HD  (1  pair  used) : 


Frequency  Response 

18  -22000  Hz 

Weight  w/o  cable 

ca.  210  g 

Design 

Open 

CEL  Instruments  CEL-231  Digital  Sound  Survey  Meter 


Range 

Low:  30-100dB 
High:  65-135dB 

Accuracy 

+  IdB 

Last  calibration 

THX™  6/21/00 

Thought  Technology  Physiological  Sensors  : 
ProComp+  Encoder  (SA7008P) 


Size  (approx.) 

81  mm  X  127mm  x  30mm  (3.2”  x  5.0”  x  ' 

Weight  (approx.) 

200g  (6.6oz) 

Channel  Bandwidth  (A,  B) 

0Hz-40Hz 

Channel  Bandwidth  (C,  D,  E,  F,  G,  H) 

0Hz-5Hz 

Sample  Rate/Channel  (A,  B) 

20  -  256  samples/second 

Sample  Rate/Channel  (C,  D,  E,  F,  G,  H) 

20  -  256  samples/second 

Supply  Voltage 

3.0V -6.5V 

Low  Battery  Warning 

3.2V  +0.2V 

Current  Consumption 

40mA  -  80mA  @  6.0V 

Accuracy 

+5% 

Data  Output  Protocol 

19.2  Kbaud,  8  Bits,  1  Stop,  No  Parity 

Battery  Life  (Alkaline) 

18  to  20  Hours  (minimum) 
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Skin  Conductance  Flex/Pro  Sensor  (SA9309M) 


Size  without  eiectrode  ieads  (approx.) 

3.5  cm  (1 .4”) 

Size  with  eiectrode  ieads  (approx.) 

15  cm  (6.0”) 

Cabie  Length  (approx.) 

127  cm  (50”) 

Weight  (approx.) 

25  g  (1  oz) 

Signai  Input  Range 

0  -  30.0  ^S 

Accuracy 

+5%  and  +0.2  )j.S 

HR/BVP  Flex/Pro  Sensor  (SA9308M) 


Size  (Approx.) 

20mm  X  34mm  x  1 0mm  (0.72”  x  1 .33”  x  0.41  ”) 

Weigh 

20g  (0.66  oz) 

Input  Range 

Unitless  quantity  displayed  as  0%  -  100% 

Accuracy 

+5% 

Temperature  Sensor  (SA9310M) 


Length  (Approx.) 

1 52cm  (60”) 

Weight 

lOg  (0.33  oz) 

Temperature  Range 

10°C-45°C  (50°F-115°F) 

Accuracy 

+1 .0°C  (+1 .8°F)  20°C  -  40°C  (68°F  -  04° F) 

MyoScan  Pro  EMG/EKG  Sensor  (SA9401M) 


Size  (Approx.) 

37mm  x  37mm  x  1 5mm  (1 .45”  x  1 .45”  x  0.60” 

Weight 

25g  (1  oz) 

Input  Impedance 

1,000,000Mil  in  parallel  with  lOpF 

Input  Range 

0-400^V,  0  -1600 

Sensitivity 

<0.1)j.Vrms 

Bandwidth 

20Hz-500Hz 

Accuracy 

+5%,  +0.3^V 
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Canon  VRC25  Video  Camera: 


Television  system 

EIA  standard  (525  lines, 60  fields) 
NTSC  color  signal 

Video  recording  system 

2  rotary  heads, 
helical  scanning  system  DV  system 
(Consumer  digital  VCR  SD  system) 

Digital  component  recording 

Television  system 

EIA  standard  (525  lines, 60  fields) 
NTSC  color  signal 

Video  recording  system 

2  rotary  heads, 
helical  scanning  system  DV  system 
(Consumer  digital  VCR  SD  system) 

Digital  component  recording 
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